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ABSTRACT 


This  report  provides  a  review  and  analysis  of  the  published  literature  on  head-mounted 
displays  (HMDs).  In  doing  so,  this  report  also  draws  heavily  from  the  basic  vision 
research  literature  in  order  to  help  provide  insight  for  future  design  solutions  for  HMDs. 
In  particular,  we  discuss  several  key  perceptual  issues  that  are  relevant  to  the  use  of 
HMDs.  The  issues  discussed  are:  (1)  brightness  and  contrast;  (2)  accommodation- 
vergence  synergy;  (3)  field  of  view;  (4)  binocular  input;  and  (5)  head  movements.  This 
review  of  the  literature  is  intended  to  anticipate  and  solve  perceptual  issues  associated 
with  two  particular  HMD  applications:  (1)  simulation  of  off-bore  sight  (OBS)  targeting 
and  (2)  full  field-of-view  out-the-window  (OTW)  simulation  for  deployable  flight 
training.  Additionally,  several  technology  issues  important  to  the  continued  development 
of  HMDs  are  discussed.  This  report  concludes  with  a  set  of  recommendations  for  the 
design  and  use  of  HMDs  for  OBS  and  OTW  flight  training  applications. 
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REVIEW  AND  ANALYSIS  OF  HELMET-MOUNTED  DISPLAYS  FOR  USE  IN  AIR  FORCE 

TRAINING  AND  SIMULATION 

SUMMARY 

This  paper  provides  a  review  and  analysis  of  some  of  the  visual  perceptual  issues  related  to  the 
use  of  helmet-mounted  displays  (HMDs)  for  training  and  simulation  applications  at  Air  Force  Research 
Laboratory,  Warfighter  Readiness  Research  Division,  Mesa,  AZ.  A  typical  HMD  system  is  presented, 
which  is  composed  of  several  parts:  an  image  generator,  a  display  device  attached  to  the  head  or  helmet, 
relay  optics,  a  combiner  if  the  imagery  is  seen  as  superimposed  on  another  display  screen  or 
superimposed  on  the  outside  world,  and  a  head-tracking  system. 

First,  although  the  focus  of  this  report  is  perceptual  issues  in  the  use  of  HMDs,  a  brief  summary 
of  current  technology  relevant  to  HMDs  is  presented.  This  summary  includes  a  review  of  currently 
available  visual  display  technology  as  well  as  some  predictions  for  future  display  technology  that  may  be 
suitable  for  helmet-mounted  use.  A  brief  summary  of  currently  available  head  tracking  technology  is  also 
included.  Next,  an  analysis  of  perceptual  issues  related  to  the  use  of  HMDs  in  two  specific  situations  is 
given:  (1)  simulation  of  off-bore  sighting,  (OBS)  and  (2)  simulation  of  out-the-window  (OTW)  viewing. 
For  simulation  of  OBS,  monocular  HMDs  may  be  used  while  the  user  views  the  OTW  display  of  a  flight 
simulator.  We  consider  the  use  of  a  monocular  HMD  to  expand  the  capabilities  of  the  current  Mobile 
Modular  Display  for  Advanced  Research  and  Training  (M2DART)  at  Air  Force  Research  Laboratory 
(AFRL),  Mesa,  AZ  to  include  features  similar  to  that  of  the  Joint  Helmet  Mounted  Cueing  System 
(JHMCS)  currently  being  fielded  in  various  airframes  (F-15,  F-16,  and  F-18).  In  this  HMD  application, 
where  only  monochrome  symbology  is  displayed,  the  technology  demands  are  fewer  compared  to  the 
OTW  HMD.  A  monochrome  image  source  with  a  relatively  narrow  field  of  view  would  be  suitable. 

Binocular,  wide  field  of  view  HMDs  provide  one  approach  for  simulating  OTW  imagery, 
particularly  in  deployable  training  applications.  A  high-resolution  display  having  a  field  of  view  greater 
than  100  degrees  is  required.  Furthermore,  motion  artifacts  in  the  display  need  to  be  minimized  as  much 
as  possible.  Additional  requirements  include:  interocular  differences  in  luminance  no  greater  than  25%,  a 
rotational  difference  no  greater  than  3.4  arcmin,  horizontal  or  vertical  differences  in  image  size  no  greater 
than  1.5%,  vertical  misalignment  less  than  5  arcminutes,  and  horizontal  misalignment  less  than  8.6 
arcminutes.  The  update  lag  between  head  movements  and  update  of  display  imagery  needs  to  be  less  than 
16.7  msec.  A  table  summarizing  these  findings  is  located  on  pages  38  and  39. 

Based  on  the  review  of  previous  HMD  research  issues  and  a  survey  of  current  technology, 
recommendations  for  future  research  projects  are  also  given: 
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(1)  Depth  of  Focus.  In  simulating  OBS  using  a  monocular  HMD  worn  in  a  flight  simulator,  the 
plane  of  focus  of  the  symbology  on  the  HMD  should  match  the  viewing  distance  to  the  display  screen  of 
the  simulator  so  that  all  images  are  in  focus.  This  will  not  be  an  issue  for  dome-type  display  systems 
assuming  the  focal  length  of  the  HMD  imagery  has  been  adjusted  to  match  the  radius  of  the  dome. 
However,  for  faceted  display  systems  such  as  the  AFRL  M2DART,  Boeing  VIDS  III,  Link  Simulation 
and  Training  Simusphere,  and  Glass  Mountain  Optics  WASP  displays,  the  distance  from  the  viewer  to  the 
display  surface  varies  with  direction  of  gaze.  The  rear-projected  imagery  and  HMD  imagery  will 
therefore  be  at  differing  depths  and  the  observer  may  not  be  able  to  maintain  focus  on  each  image 
simultaneously.  The  results  of  a  study  conducted  at  AFRL,  Mesa  to  evaluate  whether  differing  focal 
distances  will  hinder  the  feasibility  of  simulating  OBS  using  a  monocular  HMD  in  the  M2DART  are 
summarized  on  page  16. 

(2)  Accommodation- Vergence  Mismatch.  When  simulating  off-bore  sighting  with  a  monocular 
HMD  worn  in  a  flight  simulator,  or  simulating  OTW  viewing  with  a  binocular  HMD,  the  stimulus  to 
accommodation  will  be  the  imagery  of  the  HMD  and/or  display  screen  of  the  simulator.  However,  when 
the  user  changes  his  or  her  vergence  angle  to  binocularly  view  a  virtual  target  in  the  synthetic  scene,  the 
vergence  angle  can  be  mismatched  relative  to  accommodative  demand.  There  are  three  potential  effects 
of  accommodation- vergence  mismatch:  (1)  eye  strain  or  visual  discomfort;  (2)  blurring  of  imagery 
(depending  upon  display  brightness  and  contrast);  and  (3)  misperceived  size,  depth,  speed,  and/or 
distance  via  the  operation  of  perceptual  constancy. 

(3)  Binocular  Rivalry.  In  simulating  off-bore  sighting,  one  eye  views  the  reticle  and  both  eyes 
view  the  OTW  imagery.  In  this  case,  the  existence  of  interocular  differences  in  stimulus  characteristics 
has  the  potential  to  produce  binocular  rivalry.  In  viewing  OTW  imagery  with  a  binocular  HMD,  both 
eyes  will  view  the  out-the-window  scene  of  an  ongoing  mission.  Off-bore  sighting  can  be  also  be 
simulated  using  a  binocular  HMD  by  presenting  the  appropriate  symbology  to  one  eye  (or  both)  in 
addition  to  the  OTW  imagery.  In  this  case,  the  existence  of  small,  unintended  interocular  differences  in 
stimulus  characteristics  may  occur  due  to  imprecision  of  mounting  and  alignment,  which  may  also 
produce  rivalry.  Luning,  which  is  often  present  in  binocular  displays  with  partial  overlap,  is  also  thought 
to  be  caused  by  rivalry.  It  is  therefore  important  to  determine  which  stimulus  parameters  produce  or 
minimize  rivalry  and  whether  training  or  practice  can  help  mitigate  such  effects. 

(4)  Field  of  View.  This  is  primarily  an  issue  for  simulation  of  OTW  viewing  with  a  binocular 
HMD.  Field  of  view  can  have  a  significant  effect  on  performance  of  tasks  relevant  to  pilot  training.  It  is 
important  to  determine  what  size  field  of  view  is  required  and  how  field  of  view  requirements  need  to  be 
weighed  against  other  requirements,  such  as  resolution  and  binocular  overlap.  If  high  resolution  is  a 


2 


driving  requirement,  for  example,  field  of  view  must  necessarily  be  smaller.  If  depth  cues  are  important 
in  a  particular  HMD  application  then  full  binocular  overlap  with  binocular  disparity  across  the  field  of 
view  may  be  a  requirement.  This  would  also  necessitate  a  smaller  field  of  view.  Technology 
development  is  needed  to  address  this  issue. 

(5)  Binocular  Overlap  and  Convergent  versus  Divergent  Displays.  In  simulating  OTW  viewing, 
it  is  important  to  determine  the  perceptual  implications  of  using  either  convergent  or  divergent  binocular 
displays.  In  particular,  one  should  determine  how  the  convergent  versus  divergent  systems  relate  to  half¬ 
occlusions,  visual  suppression,  luning,  and  field  of  view,  and  whether  one  type  of  display  is  better  than 
the  other  or  whether  a  full-overlap  display  should  be  considered. 

(6)  Head  tracking  of  voluntary  head  movements.  In  simulating  out-the- window  viewing,  the 
display  imagery  must  be  shifted  in  a  direction  opposite  to  voluntary  head  movements  to  mimic  an 
environment-centered  frame  of  reference.  One  potential  problem  is  that  the  time  required  to  measure  the 
head  movements  with  the  head  tracking  system  and  update  the  display  creates  a  temporal  lag  that  impairs 
perception  and  places  constraints  on  the  gain  of  the  head  tracking  system.  It  is  therefore  important  to 
investigate  the  relationships  among  head-tracking  accuracy,  lag,  and  size  of  head  movement  to  determine 
an  acceptable  set  of  parameters. 

(7)  Head  tracking  of  involuntary  head  movements  during  training  or  rehearsal.  If,  during  mission 
training  or  rehearsal,  the  user  employs  an  HMD  in  an  environment  that  causes  acceleration  or  vibration, 
the  head  will  execute  involuntary  head  movements,  which,  in  turn,  elicit  a  compensatory  and  opposite 
vestibulo-ocular  reflex  (VOR)  eye  movement.  The  VOR  eye  movement  impairs  viewing  of  the  HMD 
because  the  image  moves  with  the  head,  which  renders  the  VOR  response  as  inappropriate.  One  approach 
for  correcting  inappropriate  VOR  responses  is  to  measure  the  involuntary  head  movements  with  a  head 
tracker  and  shift  the  image  on  the  HMD  by  the  same  magnitude  and  a  direction  opposite  to  the  head 
movement  (in  the  same  direction  as  the  eye  movement).  One  problem  with  this  technique  is  that  the  head 
tracker  would  likely  not  be  accurate  enough  to  fully  compensate  for  the  small  amplitude  VOR  response. 

It  is  therefore  important  to  determine  to  what  degree  HMDs  will  be  used  for  mission  training  and 
rehearsal  in  environments  involving  acceleration  or  vibration.  This  could  occur  if  mission  training  or 
rehearsal  were  conducted  on  an  aircraft  carrier  or  transport  plane  while  enroute  to  a  new  mission. 
Significant  vibration  or  acceleration  would  also  occur  if  the  HMD  were  used  in  conjunction  with  a 
moving  base  simulator  or  motion  seat. 
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INTRODUCTION 

Recent  demands  for  deployable  training  systems  and  the  adoption  of  HMDs  in  fielded  aircraft  for 
off-bore  sighting  (OBS)  have  renewed  interest  in  HMDs  for  flight  training  applications.  An  HMD 
presents  symbolic  or  pictorial  information  to  a  pilot  or  other  user  while  allowing  changes  in  head  position. 
Current  generation  HMDs  use  miniature  displays  mounted  on  a  helmet  and  relay  optics  that  present 
imagery  to  the  user's  eyes  (Beal  &  Sweetman,  1997;  Velger,  1998).  These  displays  offer  some 
advantages  over  traditional  displays,  such  as  increased  situational  awareness  in  augmented  visibility 
situations  (Velger,  1998).  HMDs  would  also  be  useful  in  mission  training  and  rehearsal  simulations  due 
to  their  portability. 

If  implemented  properly,  HMDs  can  offer  advantages  over  traditional  displays,  such  as  increased 
situational  awareness  and  ease  of  mobility.  Thus,  HMDs  are  increasingly  being  considered  for  use  in  a 
wide  variety  of  applications,  including  surgery  and  other  medical  applications,  remote  vehicle  operation, 
flight  simulation,  and  automobile  racing  (Melzer  &  Moffitt,  1997;  Velger,  1998). 

Despite  the  potential  advantages  of  HMDs,  their  development  through  the  years  has  not  been 
uniform  (Ruston,  Mon-Williams,  &  Wann,  1994;  Sheedy,  &  Bergstrom,  2002).  Although  Peli  (1998) 
notes  that  visual  effects  during  the  use  of  an  HMD  were  similar  to  those  of  a  desktop  display,  Keller  and 
Colucci  (1998)  state  that  HMDs  have  often  disappointed  real  world  users.  While  HMDs  have  been  used 
successfully  in  certain  applications,  their  use  in  other  applications  has  been  troublesome  (Velger,  1998). 
This  is  due,  in  part,  to  the  fact  that  HMDs  can  create  significant  perceptual  problems  not  found  with 
traditional  displays  (Melzer  &  Moffitt,  1997).  For  example,  Wenzel,  Castillo  and  Baker  (2002)  found 
that  aircraft  maintenance  workers  who  used  a  HMD  for  training  purposes  reported  problems  such  as 
eyestrain,  headache,  nausea,  and  dizziness.  Morphew,  Shively,  and  Casey  (2004)  found  that  self-reports 
of  nausea,  disorientation,  and  oculomotor  strain  were  greater  with  an  HMD  compared  to  a  standard 
computer  monitor  for  a  Unmanned  Aerial  Vehicle  control  task.  Hakkinen  (2004)  reports  similar 
problems  when  a  monocular  HMD  was  used  when  performing  a  text-editing  task.  Kooi  (1996)  reported 
significantly  greater  eyestrain  with  the  use  of  HMDs  compared  to  a  standard  computer  monitor. 

Simulator  sickness  has  also  been  reported  with  the  use  of  HMDs  and  can  be  caused  by  a  number  of 
factors  (Ehrlich,  1997;  Draper,  Virre,  Furness,  &  Gawron,  2001;  Draper,  Virre,  Furness,  &  Parker,  1997). 

In  order  to  address  these  issues  for  improved  future  applications  an  understanding  of  the 
perceptual  issues  underlying  these  phenomena  is  necessary.  To  that  end,  this  paper  provides  a  review  and 
analysis  of  some  of  the  visual  perceptual  issues  related  to  the  use  of  helmet-mounted  displays  in  two 
situations:  (1)  simulation  of  off-bore  sighting,  and  (2)  simulation  of  OTW  viewing.  The  published 
literature  on  HMDs  as  well  as  basic  perceptual  processes  is  reviewed  and  perceptual  issues  identified  that 
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are  relevant  to  HMD  systems  currently  available  and  likely  to  be  commercially  available  in  the  near 
future.  A  summary  of  currently  available  HMD  systems  suitable  for  use  in  simulation  of  off-bore 
targeting  and  OTW  viewing  applications  is  also  included.  We  also  provide  some  discussion  of  HMD 
components,  such  as  visual  display  technology,  head  tracking  systems,  and  optics.  However,  we  are 
primarily  concerned  with  perceptual  issues  that  may  arise  with  the  use  of  HMDs  in  specific  applications. 
For  a  more  thorough  description  of  HMD  engineering  design  and  technology  issues  see,  for  example, 
Velger,  1998  or  Melzer  and  Moffit,  1997.  Finally,  recommendations  for  future  areas  of  research  to 
investigate  certain  perceptual  issues  will  be  given. 


BASICS  OF  HMDS 

A  typical  HMD  system  is  composed  of  several  parts:  an  image  generator,  a  display  device 
attached  to  the  head  or  helmet,  relay  optics,  a  combiner  if  the  imagery  is  seen  as  superimposed  on  another 
display  screen  or  superimposed  on  the  outside  world,  and  a  head-tracking  system  (Velger,  1998). 


The  HMD  can  be  a  monocular  system  (Figure  1);  in  which  synthetic  imagery  is  presented  to  only 
one  eye  of  the  user  (the  other  eye  sees  the  real  world).  This  situation  involves  one  image  source  and  one 
set  of  optics,  therefore  the  HMD  is  relatively  lightweight.  The  use  of  monocular  HMDs  creates  the 
potential  for  perceptual  problems  such  as  binocular  rivalry  and  visual  discomfort.  The  HMD  can  also  be 
a  biocular  system,  in  which  the  same  imagery  is  presented  to  both  eyes  of  the  user  (Rogers  &  Freeman, 
1992),  or  a  binocular  system,  in  which  slightly  different  imagery  (i.e.,  imagery  with  binocular  parallax)  is 
presented  to  the  two  eyes  (Klymenko  et  ah,  1994).  Binocular  displays  have  the  advantage  of  providing 
stereoscopic  depth  information  to  the  user  but  significant  disadvantages  of  greater  complexity,  alignment, 
and  precision  mounting  issues.  They  will  also  require  the  use  of  two  separate  visual  channels  with 
slightly  different  perspectives.  An  example  of  a  binocular,  see-through  HMD  is  shown  in  Figure  2  (left) 
as  well  as  a  binocular  occluding  HMD  (right). 


Figure  1.  Left,  Monocular  see-through  (courtesy  of  Micro  vision)  and  right,  monocular  occluding 

HMD  (courtesy  of  LiteEye). 
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Figure  2.  L3  Communications  Advanced  HMD  (left,  courtesy  of  L3  Communications)  and  Kaiser  SR 
80  binocular  HMD  (right,  courtesy  of  Kaiser  Electro-Optics). 

An  ideal  HMD  would  have  high  resolution,  very  short  imagery -update  lag,  and  be  lightweight 
with  an  appropriate  center  of  mass  (Velger,  1998;  Wood,  1992).  For  OTW  simulation,  a  very  large  field 
of  view  and  large  area  of  binocular  overlap  would  be  desirable.  In  reality,  however,  trade-offs  exist 
among  these  characteristics.  Improving  one  characteristic  will  come  at  the  expense  of  another 
characteristic.  For  example,  increasing  the  field  of  view  will  require  larger  optics  and  a  corresponding 
increase  in  weight.  Center  of  mass  will  therefore  also  be  affected.  Overcoming  these  limitations  will 
require  new  technology. 

The  information  displayed  on  an  HMD  can  be  as  simple  as  an  aiming  reticle  or  alphanumeric 
symbols  or  may  consist  of  complex  full-color  moving  imagery  (Velger,  1998).  Some  HMD  applications 
use  conformal  symbology.  Conformal  symbology  refers  to  world-stabilized  symbology  that  conforms  to 
the  outside  world  (i.e.,  symbology  linked  to  an  environment-centered  frame  of  reference)  such  that  the 
symbology  changes  with  head  position.  Conformal  symbology  is  difficult  to  implement  because  the 
position  of  the  head  must  be  measured  and  the  symbology  updated  quickly  to  mimic  the  shift  in  scene 
with  changes  in  head  position  that  occurs  in  the  real  world.  Temporal  lags  in  the  measurement  of  head 
position  impair  human  performance  and  create  disorientation  and  discomfort  because  lags  delay  the 
update  of  imagery  relative  to  the  head  movement  (Long  &  Wickens,  1994;  Velger,  1998)  and  produce  a 
mismatch  between  proprioception  (sense  of  body  position)  and  vision.  Such  a  situation  exists  when  the 
HMD  displays  information  that  substitutes  for  the  real-world  view  (Ineson,  1991;  Kaye  et  al.,  1990; 

Adam,  1994;  Swenson  et  al.,  1994),  such  as  when  simulating  OTW  viewing  for  training  or  mission 
rehearsal  purposes. 
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TECHNOLOGY  ISSUES 


Visual  Displays 

Current  Technology 

There  are  many  types  of  visual  displays  that  could  potentially  be  used  in  helmet-mounted 
applications.  Miniature  monochrome  cathode  ray  tube  (CRT)  displays  have  generally  been  used  with 
HMDs  in  the  past  due  to  a  lack  of  viable  alternatives.  CRTs  do  offer  high  luminance  output,  low  cost, 
durability,  and  desirable  temporal  characteristics.  Significant  disadvantages  of  CRT  displays,  however, 
include  their  relatively  high  weight,  large  size,  and  high  power  requirements. 

Other  types  of  displays  are  now  beginning  to  replace  CRTs  for  use  in  HMDs.  The  most 
promising  visual  display  as  a  replacement  for  CRTs  for  use  in  HMDs  is  the  Liquid  Crystal  on  Silicon 
(LCoS)  display.  These  displays  are  relatively  fast,  have  good  light  output,  low  power  consumption,  low 
weight  and  small  size,  high  fill  factor,  and  relatively  high  resolution.  LCoS  chips  have  a  greater  potential 
to  achieve  high  pixel  counts  and  manufacturing  costs  are  lower  compared  to  some  other  technologies 
(McLaughlin,  2002).  Limitations  of  LCoS  displays  include  some  brightness  limitation  due  to  heating  of 
the  liquid  crystal,  color  shifts  over  time,  and  limitations  to  contrast  due  to  anisotropy  and  polarization 
coupler  efficiency.  There  are  also  some  limitations  to  switching  time  due  to  capacitance  (Pinho,  2004). 
Another  potential  disadvantage  of  LCoS  displays  in  HMD  applications  is  the  requirement  for  an 
illumination  source,  which  must  be  mounted  in  front  of  the  LCoS  panel.  This  may  impose  some 
limitations  on  the  configuration  of  the  optics  for  a  head  mounted  design.  Furthermore,  the  lamp  may 
periodically  need  replacement.  However,  the  development  of  high  output  light  emitting  diodes  (LEDs) 
alleviates  these  concerns  to  some  extent.  Current  chip  designs  also  cause  the  LCoS  pixels  to  be 
illuminated  for  an  entire  video  frame  (typically  16.7  milliseconds).  This  hold-time  increases  the  light 
output  of  the  device,  but  may  cause  motion  artifacts  such  as  blurring  or  double  images  (see  e.g., 
Winterbottom,  Geri,  &  Pierce,  2004  and  Lindholm,  Pierce,  &  Scharine,  2001).  These  artifacts  are  clearly 
undesirable  in  military  simulation  applications,  particularly  fast  jet  simulation.  These  motion  artifacts  can 
be  predicted  through  analyses  in  the  spatiotemporal  domain  if  the  properties  of  both  the  display  and  the 
observer  are  known  (Watson,  Ahumada,  &  Farrell,  1986).  An  additional  complication  for  helmet- 
mounted  applications,  particularly  OTW  applications,  is  that  the  imagery  must  move  when  the  head 
moves.  Movement  of  the  head  will  cause  significant  blurring,  or  loss  of  definition,  if  the  full  16  msec 
hold-time  is  maintained.  A  new  type  of  LCoS,  a  ferro-electric  LCoS  (FLCoS),  which  has  recently  been 
introduced  (see  for  example  http://www.crlopto.com/),  may  provide  a  solution  to  the  pixel  hold-time 
issue.  The  new  FLCoS  is  significantly  faster  than  even  the  standard  LCoS.  Provided  that  sufficient 
illumination  is  available,  it  is  possible  that  the  pixel  hold-time  on  the  FLCoS  could  be  reduced  sufficiently 
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(e.g.,  <  4  msec)  to  reduce  motion  artifacts.  Several  HMD  manufacturers  are  now  using  FLCoS  (e.g., 
Kaiser  SR80  and  SRI 00,  L3  Communications  Advanced  HMD,  and  NVIS  nVisor). 


Figure  3.  An  FLCoS  display  panel  (Used  with  permission  of  CRL  Opto  Ltd.). 

Active  matrix  liquid  crystal  displays  (AMLCDs)  also  have  the  advantage  of  low  weight  and  small 
size  and  have  good  image  quality  and  color.  However,  they  also  require  a  lamp  and  typically  have 
temporal  characteristics  that  are  too  slow  for  applications  involving  moving  imagery,  such  as  fast  jet 
simulation.  Head  motion  will  also  result  in  significant  blurring  with  LCD  displays.  LCDs  also  have  a 
pixel  hold  time  of  one  frame,  resulting  in  the  additional  temporal  issues  described  for  the  LCoS  displays. 
Additional  drawbacks  of  LCD’s  include  some  brightness  limitation  due  to  heating  of  the  liquid  crystal, 
color  shifts  over  time,  and  limitations  to  contrast  due  to  anisotropy  and  polarization  coupler  efficiency. 
There  are  also  some  limitations  to  switching  time  due  to  capacitance  (Pinho,  2004).  These  are  similar  to 
the  limitations  to  LCoS  displays  -  both  use  liquid  crystal  technology. 

Electroluminescent  displays  (or  active  matrix  EL  displays)  have  high  luminous  efficiency  and 
brightness  (300-500  ft-L)  as  well  as  high  contrast  ratios  1:50-1:100  (Monarchic  et  ah,  1995).  These 
devices  use  alternating  current  to  excite  a  thin-film  phosphor.  The  phosphor  layer  is  sandwiched  between 
insulating  layers  and  deposited  on  a  glass  substrate  with  transparent  electrodes  (Post,  1999).  Miniature 
devices  have  been  manufactured  for  head-mounted  use,  but  HMDs  that  utilize  this  display  technology  are 
not  widely  available. 

Virtual  retinal  displays  (VRDs)  have  been  developed  by  Micro  vision  (http://www.mvis.com/)  and 
are  commercially  available.  This  type  of  display  presents  images  to  the  visual  system  by  scanning  the 


images  directly  onto  the  retina  using  a  diode  laser  light  source  and  thus  eliminating  the  display  screen. 

The  potential  advantages  of  such  a  system  are  increased  resolution,  contrast,  brightness,  and  color  quality, 
and  decreased  weight  (Businessweek  Online,  1999).  The  bright  high-resolution  image  source  can  be 
located  off-helmet  and  its  image  presented  to  the  helmet  and  pilot  via  fiber-optic  image  guides  (Naor  et 
ah,  1987;  Thomas  et  ah,  1989;  Velger,  1998,  pp.  109).  One  disadvantage  of  this  display  device  for 
helmet-mounted  applications  is  the  small  exit  pupil.  Slippage  of  the  helmet  or  large  amplitude  eye 
movements  may  cause  portions  of  the  image  to  disappear  from  view.  Resolution  of  current  VRDs 
(800x600)  are  sufficient  for  display  of  symbology  in  monocular  applications,  but  would  need  to  be 
increased  for  binocular  OTW  applications  with  complex  imagery. 

Digital  Light  Processing  (DLP)  display  devices  (or  Digital  Micro-Mirror  Display  or  DMD) 
created  by  Texas  Instruments  (http ://www. dip . com/)  could  also  be  used  in  head-mounted  applications.  A 
DMD  display  operates  using  tiny  addressable  micro-mirrors.  These  mirrors  pivot  between  two 
orientations  to  either  reflect  light  out  through  the  lens  to  the  observer,  or  reflect  it  into  an  absorptive 
material.  The  micro-mirrors  can  be  switched  at  high  rates  of  speed  (up  to  several  thousand  times  per 
second).  The  rapidly  moving  mirrors  are  used  to  modulate  brightness  by  varying  the  amount  of  time  the 
light  is  displayed  during  the  refresh  period  (pulse-width  modulation).  DLPs  have  the  highest  brightness 
capability  of  all  the  current  digital  technologies  and  require  the  simplest  optics.  However,  the  number  of 
pixels  that  can  be  placed  on  one  chip  may  be  limited  and  they  are  currently  more  expensive  to 
manufacture  (Pinho,  2004).  Furthermore,  the  same  temporal  issues  apply  to  DLPs  as  noted  for  the  LCoS 
and  LCD  displays. 

As  noted  above,  most,  if  not  all,  currently  available  digital  displays  exhibit  temporal  artifacts, 
which  become  quite  evident  for  rapidly  moving  imagery.  Both  the  LCoS  displays,  and  current  DLPs,  for 
example,  are  designed  to  display  one  video  frame  for  an  entire  refresh  period  (typically  16  msec).  In 
addition,  many  current  DLP  projectors  are  field  sequential,  so  multiple  images  of  different  colors  may  be 
displayed  during  the  course  of  one  frame.  Field  sequential  displays  may  exhibit  color  breakup  - 
particularly  where  the  pulse  width  modulation  of  each  of  the  colors  is  spread  throughout  the  entire  16.7 
msec  frame.  For  HMDs,  where  weight  is  a  significant  concern,  a  fast  digital  display  (whether  DLP  or 
FLCoS)  is  likely  to  be  single  panel  color  sequential.  Additional  development  work  is  needed  to  reduce 
this  and  other  temporal  artifacts  in  these  types  of  digital  displays,  particularly  for  HMDs,  where  head 
motion  may  exacerbate  the  problem.  These  artifacts  can  be  predicted  when  the  temporal  characteristics, 
and  other  properties  of  the  display,  are  known  (Lindholm  &  Scharine,  2000;  Post,  Nagy,  Monnier,  & 
Calhoun,  1998).  Post  et  al  (1998),  for  example,  developed  an  equation  that  can  be  used  to  predict  the 
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display  refresh  rate  necessary  to  reduce  color  breakup  to  below  a  detectable  level  for  a  given  target 
luminance  level,  contrast,  and  rate  of  motion. 

Future  Technology 

A  recent  development  in  display  technology  is  the  laser  projector.  The  laser  projector  is  a  proof- 
of-concept  display  device  currently  under  development  by  Evans  and  Sutherland  Inc.  (E&S)  and  Air 
Force  Research  Laboratory  (AFRL).  The  goal  of  this  E&S/AFRL  development  effort  is  to  develop  a  true 
20/20  immersive  visual  display  for  simulation  and  Distributed  Mission  Operations  (DMO).  The 
technology  driving  the  increased  resolution  of  this  display  device  is  the  Grating  Light  Valve  (GLV) 
spatial  light  modulator  developed  at  Silicon  Light  Machines  in  San  Jose,  CA.  The  GLV  is  a  high-speed 
linear  array,  parallel  scan  micro-electro-mechanical  system  (MEMS)  device  that  is  currently  capable  of 
scanning  up  to  5120  x  1024  images.  An  Ultra  GLV  capable  of  scanning  5120  x  4096  pixel  images  is  in 
development  for  an  AFRL  and  Air  Combat  Command  (ACC)  Advanced  Technology  Demonstration 
(ATD)  in  2005.  Light  sources  illuminating  the  GLV  are  red,  green,  and  blue  lasers.  Additional 
information  on  this  technology  can  be  found  in  Winkler  and  Surber  (2001).  The  laser  projector  offers 
very  high  resolution  and  excellent  temporal  qualities.  The  current  size  of  the  laser  projector  is  too  large 
for  helmet  mounted  applications,  however  further  development  of  laser  illumination  technologies, 
development  of  a  color  sequential  device,  and  improved  packaging  over  the  next  several  years  could 
permit  the  eventual  design  of  an  ultra-high  resolution  laser  display  based  HMD.  Similar  technologies  are 
under  development  at  Kodak  and  the  Institute  of  National  Optics  (INO). 

Another  promising  display  technology  is  the  flexible  flat  panel.  These  displays  are  currently 
under  development  by  a  number  of  organizations  (Crawford,  2004).  A  device  of  this  type  would  offer 
numerous  advantages  over  current  display  devices  and  would  remove  many  of  the  technological  hurdles 
limiting  factors  such  as  weight,  field  of  view,  and  resolution.  These  devices  would  be  self-illuminating 
thus  avoiding  the  packaging  limitations  created  by  the  requirement  for  a  separate  light  source.  Another 
substantial  advantage  is  that  the  device  would  flexible  or  moldable.  Although  collimating  optics  would 
still  be  required  to  produce  an  image  that  appeared  to  be  at  a  distance  from  the  observer,  the  image  source 
could  be  shaped  for  a  curvature  best  fitting  the  requirements  of  the  collimating  optics.  The  elimination  of 
the  light  sources  would  also  reduce  the  weight  and  size  of  the  device. 

Another  technology  that  would  not  require  a  separate  light  source  is  the  organic-light-emitting- 
diode  (OLED)  display.  These  devices  consist  of  a  thin  transparent  layer  of  organic  semiconductors  or 
polymers,  typically  sealed  between  two  substrates,  usually  glass  or  plastic.  The  front  electrode  of  an 
OLED  display  is  transparent  and  the  rear  electrode  is  reflective.  Applying  a  voltage  across  the  electrodes 
causes  the  organic  layer  to  emit  light  (Post,  1999).  It  therefore  does  not  require  a  separate  light  source. 
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simplifying  weight  and  configuration  concerns.  These  devices  have  been  limited  in  their  light  output, 
response  time,  and  resolution,  but  are  making  significant  advances.  OLED  displays  are  already  beginning 
to  appear  in  a  variety  of  products  such  as  cell  phones  and  PDAs.  An  OLED  display  of  sufficiently  high 
resolution,  brightness,  and  decreased  response  time  would  be  a  prime  candidate  for  use  in  a  HMD. 

A  novel  approach  for  potentially  producing  a  wide  field  of  view  visual  display  for  deployable 
training  applications  is  the  helmet-mounted  conjugate  optical  system  (CODS),  a  retro-reflective  display 
system.  This  system  is  described  in  Geri,  Arrington,  and  Miller  (1999)  and  an  initial  analysis  of  technical 
and  perceptual  issues  is  presented.  Briefly,  this  system  consists  of  a  head-mounted  display  that  projects 
an  image  in  front  of  the  observer  via  a  head-mounted  beam  splitter.  The  imagery  is  then  reflected  from  a 
screen  coated  with  retro-reflective  material  located  at  some  distance  from  the  observer.  The  advantage  of 
such  a  system  is  that  it  provides  collimated  imagery  -  that  is  the  imagery  appears  to  be  located  at  some 
distance  from  the  observer.  The  optics  of  the  system  described  by  Geri  et  al  (1999)  are  also  relatively 
simple  compared  to  a  typical  HMD.  A  similar  but  lighter  and  more  advanced  display  system,  a  helmet 
mounted  projection  display  (HMPD),  has  been  developed  at  the  Optical  Diagnostic  and  Applications 
Laboratory  (ODA  Lab)  at  the  University  of  Central  Florida.  They  have  also  tested  and  developed  a 
variety  of  materials  for  the  required  retro-reflective  surface.  Additional  description  of  this  work  is 
available  at  http://odalab.ucf  edu/. 

For  an  image  plane  located  farther  away  than  the  retro-reflective  screen  the  appearance  of  the 
system  is  described  as  similar  to  that  of  looking  through  a  window.  The  limitations  of  this  system, 
however,  are  similar  to  that  of  currently  available  HMDs.  A  higher  resolution  and  wider  field  of  view  is 
needed  for  an  acceptable  deployable  display  system.  This  requirement  necessitates  either  a  display  with  a 
substantially  greater  number  of  pixels  or  the  tiling  of  multiple  displays.  Although  the  optics  of  the  device 
described  by  Geri  et  al  (1999)  and  the  ODA  lab  may  be  simpler  compared  to  currently  available  HMDs,  a 
larger  field  of  view  would  require  larger  beam  splitters  and,  if  tiled  displays  are  used,  additional  light 
engines. 

Optics 

The  optical  elements  used  in  HMDs  serve  to  direct  images  from  the  display  to  the  eye.  Various 
types  of  optics  have  been  developed  and  different  types  of  optical  designs  result  in  varying  trade-offs  in 
weight,  image  quality,  field-of-view,  and  light  throughput.  A  considerable  amount  of  effort  and  expense 
goes  into  the  design,  consideration,  and  selection  of  optical  components  for  HMDs.  The  focus  of  this 
review,  however,  is  on  the  perceptual  difficulties  that  may  be  encountered  during  the  use  of  HMDs  in 
particular  applications.  For  a  more  detailed  description  of  various  optical  designs  Velger  (1998),  and 
Melzer  and  Moffit  (1997)  offer  good  descriptions. 
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Head  Tracking  Systems 

Most  applications  with  HMDs  will  require  the  use  of  head  trackers  to  update  the  position  of 
imagery  as  the  head  moves.  For  OBS  applications,  the  position  of  symbology  may  need  to  be  moved  and 
the  position  of  an  aiming  reticle  will  need  to  be  specified  for  targeting  purposes.  The  speed  and  accuracy 
of  position  updates  are  therefore  very  important.  In  OTW  applications  the  speed  and  accuracy  of  these 
updates  is  even  more  critical  because  any  lag  in  the  head  tracking  and  image  update  will  be  immediately 
obvious  to  the  observer. 

Head-position  tracking  devices  may  measure  3  degrees  of  freedom  or,  more  typically,  measure 
six  degrees  of  freedom  (DOF)  of  head  position,  including  three  angular  orientations  of  roll,  pitch,  yaw, 
and  three  linear  positions  of  x,  y,  z  (Velger,  1998,  pp.  78,  143).  Measurement  of  3  degrees  of  freedom  is 
sufficient  for  applications  where  exact  positioning  of  imagery  relative  to  head-movement  is  not  required. 
However,  this  type  of  system  may  not  measure  head  roll  -  cocking  the  head  slightly  to  the  left  or  right  for 
example. 

There  are  several  types  of  head  tracking  devices.  These  include  magnetic  (A/C  or  D/C),  electro- 
optical,  acoustic,  and  inertial  (or  combinations  of  these).  Electromagnetic  trackers,  use  an  electro-magnet 
to  create  a  magnetic  field.  The  location  of  a  sensor,  which  is  mounted  on  the  helmet,  can  then  be  tracked 
as  it  moves  through  the  magnetic  field  (Velger,  1998,  pp.  147).  Electro-optical  trackers  operate  using 
remote  measurement  by  camera  of  the  positions  in  space  of  LEDs  mounted  on  the  helmet  (Velger,  1998, 
pp.  158;  Toker  &  Velger,  1991).  Acoustic,  or  ultrasonic,  trackers  are  based  on  time  of  flight  or  phase 
coherence,  which  measure  distances  between  emitters  and  receivers  to  compute  head  position  (Velger, 
1998,  pp.  165;  Meyer,  Applewhite  &  Biocca,  1992).  These  head  trackers  are  commercially  available 
from  several  sources.  Intersense  Inc.  (www.intersense.com)  manufactures  the  IS-900  6-DOF  acoustic- 
inertial  tracker,  and  the  lesser  expensive  IS-300  3-DOF  inertial  tracker.  Polhemus  (www.polhemus.com) 
manufactures  the  Liberty  A/C  electro-magnetic  6-DOF  tracker,  the  Patriot  6-DOF  A/C  electro-magnetic 
tracker,  and  the  Fastrak  A/C  electro-magnetic  tracker.  Ascension  Technology  Corporation 
(www.ascension-tech.com)  manufactures  the  LaserBird  6-DOF  electro-optical  tracker.  Flock  of  Birds  DC 
magnetic  6-DOF  tracker,  and  PCI  Bird  DC  magnetic  6-DOF  tracker. 

The  various  types  of  head  trackers  have  differing  advantages  and  disadvantages.  Generally,  the 
more  expensive  systems  will  have  higher  update  rates  and  may  have  more  sensors  for  improved  speed 
and  accuracy  (for  example,  the  Polhemus  Liberty  has  8  sensors  and  an  update  rate  of  240  Hz,  the  Patriot 
is  a  2  sensor  60  Hz  system).  The  sophistication  of  the  motion  prediction  and  noise  filtering  algorithms 
may  also  vary  from  system  to  system.  Frequently,  a  Kalman  filter  is  used  to  extrapolate  head  position 
several  milliseconds  ahead  of  time  to  improve  head  tracking  (Velger,  1998).  Additional  sensors  may  also 
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be  used  for  more  accurate  head  tracking  (Velger,  1998).  The  electro-optical  trackers  have  less  temporal 
lag  than  the  electromagnetic  trackers.  Electro-magnetic  tracking  systems  may  also  be  affected  by  nearby 
metal  objects  and  electromagnetic  radiation  (Ferrin,  1991).  However,  the  electro-optical  trackers  require 
a  direct  line-of-sight  and  large  field  of  view  (Velger,  1998,  pp.  80).  Range  of  motion  may  also  have  some 
limitations  depending  on  the  placement  of  the  LEDs  and  cameras.  The  ultrasonic  trackers  could 
potentially  have  problems  due  to  contamination  by  external  acoustic  noise  (Velger,  1998,  pp.  165).  The 
acoustic  sensors  also  require  very  precise  placement  to  maintain  accuracy  and  likely  require  professional 
installation.  Inertial  trackers  have  the  advantage  that  they  are  self-contained  and  do  not  require  a  separate 
signal  source  or  sensor;  however,  they  will  have  a  slower  response  and  may  also  tend  to  drift.  Adding  an 
acoustic  sensor  (such  as  with  the  IS-900)  allows  the  inertial  sensor  drift  to  be  corrected. 

Anderson,  Vrana,  Riegler,  and  Martin  (2002)  evaluated  several  types  of  head-tracking  systems 
for  use  with  the  Night  Vision  Training  System  developed  at  AFRL,  Mesa  and  describe  the  conditions 
under  which  one  type  of  head-tracking  system  may  have  an  advantage  over  another.  Inertial  head 
trackers  were  recommended  for  enclosed  areas  where  the  placement  of  additional  sensors  is  not  possible. 
Optical  head-trackers  were  recommended  under  conditions  where  no  instruments,  switches,  or  visual 
displays  are  located  above  the  observers’  heads,  and  in  situations  where  it  is  not  required  that  technicians 
need  to  work  around  the  observer  as  they  will  block  the  optical  sensors  tracking  the  observers’  head 
movements.  Magnetic  trackers  were  recommended  if  operating  space  is  minimal.  Magnetic  trackers 
have  an  advantage  over  inertial  because  they  provide  orientation  data  (heading,  pitch,  roll)  as  well  as  X, 

Y,  and  Z  coordinates.  DC  trackers  may  work  better  in  cases  where  more  metal  is  present.  A  magnetic 
map  can  be  obtained  for  custom  fits  to  the  working  area.  If  large  pieces  of  metal  are  introduced  following 
installation  a  new  magnetic  map  may  need  to  be  generated  to  maintain  the  accuracy  of  the  head-tracker. 
However,  Vision  Systems  International  (VSI,  San  Jose,  CA,  http://www.vsi-hmcs.com/  -  personal 
communication)  has  indicated  that  the  magnetic  head  trackers  are  fairly  robust  and  the  introduction  of 
relatively  small  metal  objects  does  not  significantly  affect  the  accuracy  of  the  head  trackers  used  in 
conjunction  with  the  Joint  Helmet  Mounted  Cueing  System  (JHMCS). 

To  accommodate  a  wide  range  of  training  tasks  the  following  ranges  of  head  movements  should 
be  measurable  with  a  head  tracking  system:  angular  azimuth  (180  deg),  elevation  (130  deg),  and  roll  (120 
deg),  with  an  accuracy  of  about  1-2  mrad  on-bore  sight  and  2-6  mrad  at  10  deg  eccentricity,  and  linear 
displacements  in  the  vertical  (450mm),  horizontal  (400  mm),  and  fore/aft  (540  mm)  axes  (Velger,  1998, 
pp.  144-145). 
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SPECIFIC  HMD  APPLICATIONS 


Off-Bore  Sighting 

A  primary  application  for  monocular  HMDs  at  AFRL  is  for  simulating  OBS,  i.e.,  sighting  and 
designating  targets  located  off-axis  relative  to  the  aircraft)  in  a  flight  simulator  for  training  purposes.  This 
will  allow  the  Mesa  Distributed  Mission  Operations  (DMO)  testbed  to  stay  current  with  upgrades  to 
fielded  aircraft  as  the  JHMCS  is  currently  being  tested  in  the  F-15,  F-16,  and  F-18.  The  JHMCS  allows  a 
pilot  to  track  and  designate  targets  for  off-bore  missile  targeting.  In  short,  the  pilot  can  shoot  whatever 
he/she  looks  at.  This  capability  allows  the  pilot  to  designate  targets  over  a  much  wider  field  of  view 
(limited  only  by  the  sensor  capabilities  of  the  missile  targeting  system).  Currently,  target  designation  can 
only  occur  within  the  limited  field  of  view  of  the  head-up  display  (HUD).  OBS  is  therefore  quite  clearly 
a  significant  advantage  in  air-to-air  engagements.  In  addition  to  OBS  capability,  the  JHMCS  also  allows 
the  pilot  to  designate  ground  targets,  view  symbology,  such  as  attitude,  altitude,  air  speed,  etc.  that  would 
normally  be  presented  on  the  HUD,  and  obtain  information  regarding  target  locations  from  other  aircraft 
via  data  link. 


Figure  4.  Joint  Helmet  Mounted  Cueing  System  (courtesy  of  VSI,  LLC). 

Requirements  for  a  device  such  as  the  JHMCS  for  training  and  simulation  purposes  may  be  quite 
different  from  those  in  the  real  aircraft.  For  example,  in  the  aircraft,  the  brightness  and  contrast  of  the 
JHMCS  needs  to  be  great  enough  to  produce  imagery  that  is  visible  against  a  bright  sky,  which  can  be  at 
levels  on  the  order  of  10,000  fL  (Velger,  1998).  This  is  not  a  requirement  for  simulation  where 
luminance  levels  do  not  typically  exceed  30  fL  with  current  CRT-based  rear  projection  imagery. 
However,  similar  contrast  levels  between  the  HMD  symbology  and  background  imagery  in  the  simulator 
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relative  to  the  real  world  would  be  desirable.  For  simulation  purposes  the  OBS  HMD  would  first  have  to 
be  integrated  with  current  visual  displays  used  for  display  of  OTW  imagery  as  well  as  the  computers  used 
for  sensor  simulation,  and  aircraft  orientation  and  motion  in  order  to  correctly  model  the  designation  of 
targets  using  the  HMD  targeting  symbology  and  correctly  update  attitude,  heading,  and  air  speed 
symbology.  Any  head  tracking  lag,  bore  sight  errors,  or  other  difficulties  that  are  present  in  the  real 
JHMCS  should  be  duplicated  for  training  purposes  in  the  simulated  system.  These  issues  may  impose 
some  unique  engineering  and  integration  challenges.  However,  our  focus  here  will  be  on  the  perceptual 
difficulties  that  may  be  encountered  with  the  incorporation  of  this  type  of  helmet-mounted  device  into  a 
current  visual  display  system,  specifically  the  Mobile  Modular  Display  for  Advanced  Research  and 
Training  (M2DART). 

Perceptual  Issues  in  Monocular  HMD  Applications 

Depth  of  focus 

The  M2DART  is  a  full  360  degree  field  of  view  faceted  display  system  used  for  DMO  training 
and  development  at  AFRL,  Mesa  (see  Wight,  Best,  and  Peppier,  1998  for  a  full  description).  This  display 
system  consists  of  eight  flat  display  screens,  with  imagery  rear-projected  (see  Fig.  5).  Because  it  is  a 
faceted  display  system,  the  distance  from  the  eye-point  to  the  display  screen  varies  as  the  observer’s  head 
turns.  This  variation  in  viewing  distance  may  create  visibility  problems  for  either  the  HMD  imagery  or 
the  simulated  OTW  imagery  because  the  observer  may  not  be  able  to  focus,  or  accommodate  to  both 
images  simultaneously.  The  HMD  symbology,  the  simulated  OTW  imagery,  or  both,  may  be  blurred. 

The  extent  to  which  blurring  may  occur  depends  on  the  depth  of  focus  of  the  pilots  or  other  users 
observing  the  HMD  and  rear-projected  imagery.  Depth  of  focus  refers  to  the  range  of  distances  in  image 
space  within  which  an  image  appears  in  sharp  focus,  and  is  specified  in  terms  of  diopters.  An  analogous 
spatial  interval,  given  in  meters,  is  referred  to  as  depth  of  field  (Ogle  &  Schwartz,  1959).  Depth  of  field 
can  be  calculated  using  the  formula!)  =  1!  F  ,  where  D  is  depth  of  field  (meters)  and  F  is  depth  of  focus 
(diopters).  Objects  varying  in  depth,  such  as  variation  in  depth  between  the  HMD  and  simulated  OTW 
imagery,  to  an  extent  greater  than  a  pilot  or  other  observer’s  depth  of  focus  will  cause  perceived  blurring. 
In  the  M2DART,  this  variation  can  be  as  much  as  19  inches  if  the  focus  distance  of  the  HMD  imagery  is 
set  equal  to  that  of  the  nearest  distance  between  eye-point  and  display  screen. 

A  similar  problem  could  be  encountered  with  other  faceted  display  systems,  such  as  the  VIDS 
available  from  The  Boeing  Corporation  (http://www.boeing.com/defense- 
space/aerospace/training/visual/visual.html),  the  Simusphere  available  from  L3  Communications 
(http://www.link.com/products/simusphere.pdf),  or  the  WASP  available  from  Glass  Mountain  Optics 
(http://www.glassmountain.com/wasp.htm).  However,  the  variation  in  distance  between  image  planes 
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will  differ  on  each  of  these  systems.  The  greater  the  distance  between  the  HMD  image  plane  and  the 
simulated  OTW  image  plane,  the  greater  the  potential  for  perceived  blurring  of  the  imagery.  Difference 
in  distance  between  image  planes  would  not  be  an  issue  for  a  dome  type  display,  such  as  that  available 
from  Barco  (Barco  Simulation  Inc.),  so  long  as  the  focal  distance  of  the  HMD  is  matched  to  the  radius  of 
the  dome. 

Other  factors  affecting  depth  of  focus  include  the  brightness  and  contrast  of  both  the  HMD 
imagery  and  the  rear-projected  simulated  OTW  imagery  and  display  resolution.  Display  brightness  will 
affect  pupil  size,  which  in  turn  will  influence  depth  of  field.  A  brighter  adapting  luminance  will  decrease 
pupil  size  and  increase  depth  of  field,  thereby  decreasing  the  likelihood  of  blurring  of  imagery  on  either 
display.  For  example.  Ogle  and  Schwartz  (1959)  report  that  for  each  mm  of  increase  in  pupil  size 
(ranging  from  2.5  to  8.0  mm),  depth  of  focus  decreased  by  0.12  diopters.  Conversely,  a  greater  degree  of 
resolution  will  lead  to  a  smaller  the  depth  of  field  (Campbell,  1957;  Ogle  &  Schwartz,  1959).  Ogle  and 
Schwartz  (1959)  showed  that,  using  Snellen  notation,  for  each  step  of  increase  in  target  resolution,  the 
total  depth  of  focus  decreased  by  0.35  diopters. 

AFRL,  Mesa  conducted  a  study  to  determine  whether  or  not  depth  of  focus  may  be  an  issue  for 
simulating  a  JHMCS  type  device  inside  a  faceted  display  system  such  as  the  M2DART,  as  well  as 
establish  a  focal  plane  distance  for  the  HMD  imagery  that  will  minimize  any  blurring  that  may  be  present 
(Winterbottom,  Patterson,  Pierce,  Covas,  &  Winner,  2005).  The  results  of  this  study  showed  that  depth  of 
focus  would  not  be  an  issue  for  displays  with  relatively  low  resolution.  For  displays  with  relatively  high 
resolution  (e.g.,  1  arcminute/pixel)  and  the  range  of  distances  possible  in  the  M2DART,  however,  it  was 
possible  to  exceed  depth  of  focus,  causing  some  noticeable  blurring  of  imagery.  It  was  recommended  that 
the  focus  distance  of  a  monocular  HMD  integrated  with  the  M2DART  be  set  to  the  optical  midpoint  of 
the  nearest  and  furthest  viewing  distances  possible  in  the  M2DART  to  minimize  the  potential  for  blurring 
when  viewing  the  two  displays  simultaneously. 
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Figure  5.  Air  Force  Research  Laboratory’s  M2DART. 
Accommodation-vergence  mismatch 

When  viewing  a  see-through  monocular  HMD  in  a  flight  simulator,  the  stimulus  to 
accommodation  could  be  either  the  imagery  of  the  HMD  or  the  display  screen  of  the  simulator.  The 
stimulus  to  accommodation  could  therefore  be  different  than  the  distance  to  which  the  observer  is  verged. 
There  are  three  possible  effects  of  accommodation-vergence  mismatch:  (1)  eye  strain  or  visual 
discomfort;  (2)  blurring  of  HMD  symbology  or  simulator  imagery  (affected  by  stimulus  contrast  and 
brightness);  (3)  misperceived  size,  depth,  speed,  and/or  distance  via  the  operation  of  perceptual 
constancy.  On  this  latter  point,  it  should  be  noted  that  perceived  size,  depth,  and  speed  depends  upon  the 
visual  system  scaling  or  re-calibrating  retinal  size,  retinal  disparity,  or  retinal  speed  for  changes  in 
viewing  distance,  which  is  called  perceptual  constancy.  To  the  degree  to  which  accommodation  and 
vergence  provide  the  visual  system  with  proprioceptive  information  about  viewing  distance,  then  changes 
in  accommodation  or  vergence  can  affect  perceived  size,  depth,  or  speed  (see  Patterson  &  Martin,  1992). 

For  example,  consider  the  perception  of  size.  When  an  observer  moves  closer  to  an  object  retinal 
image  size  increases,  and  in  order  for  the  observer  to  correctly  perceive  that  the  objecf  s  actual  size  did  not 
vary  (which  is  called  size  constancy),  the  visual  system  likely  combines  information  about  visual  angle 
and  viewing  distance  (Foley,  Ribeiro-Filho,  &  Da  Silva,  2004).  For  example,  Peli  (1990)  notes  that  the 
perceived  size  of  imagery  on  a  monocular,  occluding,  HMD  appears  to  change  depending  on  the  distance 
of  the  surface  for  which  the  opposite  eye  is  accommodated.  Wetzel,  Pierce,  and  Geri  (1996)  varied 
viewing  distance  for  alternately  presented  stimuli  and  asked  observers  to  adjust  the  size  of  the  comparison 
stimuli  so  that  the  two  were  equal.  They  found  that  stimuli  presented  at  a  near  distance  were  generally 
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judged  to  be  roughly  15-20%  smaller  than  stimuli  presented  at  the  far  distance.  Measurements  of 
vergence  angle  and  accommodation  taken  during  the  observers’  judgments  of  the  size  of  the  stimuli 
indicated  that  change  in  vergence  angle  was  strongly  related  to  the  change  in  perceived  size. 

Consider  also  the  perception  of  stereoscopic  depth,  which  is  the  perception  of  depth  based  upon 
binocular  parallax  (also  called  binocular  disparity).  When  an  observer  moves  closer  to  a  set  of  objects 
that  vary  in  depth,  binocular  disparity  increases.  In  order  for  an  observer  to  correctly  perceive  that  the 
actual  depth  relations  among  the  stimuli  did  not  vary  (which  is  called  depth  constancy),  the  visual  system 
likely  combines  information  about  binocular  disparity  and  viewing  distance  (Patterson,  Moe  &  Hewitt, 
1992;  Ono  &  Comerford,  1977;  Ritter,  1977).  An  analogous  phenomenon  may  occur  with  perceived 
speed.  When  an  observer  moves  closer  to  a  laterally  moving  object,  the  speed  of  the  object's  retinal 
image  increases,  and  in  order  for  the  observer  to  correctly  perceive  that  the  actual  speed  of  the  object  did 
not  change  (which  is  called  speed  constancy),  the  visual  system  may  combine  information  about  retinal 
speed  and  viewing  distance.  In  this  case,  however,  the  existence  of  speed  constancy  is  controversial. 
Some  authors  (Pierce  &  Geri,  1997;  Wallach,  1939;  Zohary  &  Sitting,  1993)  have  found  evidence  for 
speed  constancy,  at  least  under  some  conditions,  but  other  authors  (McKee  &  Welch,  1989)  have  not. 

Misaccommodation  may  in  fact  be  a  common  occurrence  when  viewing  virtual  displays 
superimposed  upon  a  real  scene  or  a  secondary  displayed  image.  Edgar,  Pope,  and  Craig  (1993),  for 
example,  reported  that  many  observers  tended  to  misaccommodate  when  viewing  virtual  displays 
superimposed  upon  a  real  scene.  In  this  case,  the  virtual  imagery  consisted  of  text  lettering  projected 
monocularly  in  front  of  the  observer  via  a  beam  splitter  (a  condition  similar  to  that  which  may  occur  with 
monocular  OBS).  Observers’  accommodative  state  was  measured  using  a  laser  optometer  as  they  viewed 
a  distant  building  through  a  window.  Accommodative  state  was  measured  while  observer  viewed  only 
the  building  through  the  window,  and  when  both  the  text  and  building  were  viewed.  The  results  indicated 
that  observers’  accommodative  state  tended  to  drift  towards  the  resting  state,  away  from  infinity  focus, 
when  both  the  text  and  building  were  viewed.  Edgar,  Pope,  and  Craig  also  go  on  to  comment  that  such 
misaccommodation  may  lead  to  misperceived  size  and  distance. 

In  summary,  with  monocular  HMDs  there  is  clearly  a  potential  for  a  mismatch  between 
accommodative  demand  and  vergence.  The  most  likely  symptom  of  this  mismatch  is  eyestrain  or  visual 
discomfort.  Misperceived  size,  distance,  or  speed  may  also  occur.  These  effects  could  have  some 
training  effectiveness  implications  so  it  is  important  to  determine  whether  these  conditions  occur  when  a 
monocular  HMD  is  integrated  with  another  display  system  for  training  applications.  Minimizing  the 
difference  in  distance  between  HMD  focus  distance  and  the  OTW  imagery  plane  may  minimize  the 
potential  for  this  mismatch. 
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Binocular  rivalry 

With  see-through  monocular  HMDs,  synthetic  symbology  is  presented  to  one  eye  of  the  user  via 
the  HMD,  and  the  other  eye  views  a  synthetic  or  real-world  scene,  thus  interocular  differences  may  be 
created  in  brightness,  contrast,  color,  flicker,  shape,  and  accommodative  demand  (Velger,  1998,  pp.  25). 
When  the  two  eyes  view  different  images,  a  condition  known  as  binocular  rivalry  may  ensue.  Binocular 
rivalry  is  generally  defined  as  a  state  of  competition  between  the  eyes  such  that  one  eye  inhibits  the  visual 
processing  of  the  partner  eye.  The  visibility  of  the  images  delivered  to  the  two  eyes  fluctuates  during 
rivalry  and  the  images  projected  to  the  two  eyes  rarely  are  visible  at  the  same  time. 

Hakkinen  (2003)  states  that  monocular  HMD  displays  produce  a  common  problem  of  rivalry  and 
perceptual  instability  and  should  be  used  only  briefly  (1-5  minutes).  However,  this  finding  is  for 
occluding  monocular  displays,  where  each  eye  is  viewing  completely  different  imagery.  Peli  (1990)  also 
documented  the  occurrence  of  binocular  rivalry  for  an  early  version  of  a  monocular  occluding  HMD. 
Hershberger  and  Guerin  (1975)  documented  the  occurrence  of  rivalry  during  the  use  of  a  monocular 
HMD  with  low  transparency  (10%).  The  results  of  their  study  suggested  that  ambient  scene  complexity, 
HMD  resolution,  HMD  luminance,  HMD  accommodation  distance,  and  ambient  scene  luminance  were 
significant  factors  contributing  to  the  occurrence  of  rivalry. 

For  applications  where  the  monocular  HMD  is  see-through  with  a  high  degree  of  transparency 
however,  it  is  unclear  at  this  point  whether  binocular  rivalry  occurs  to  a  significant  extent.  This  is  the 
case  for  the  JHMCS,  where  the  left  eye  views  the  OTW  scene  (whether  real  or  simulated)  while  the  right 
eye  views  both  the  HMD  imagery  and  the  OTW  scene  combined.  This  is  also  the  case  for  the  Integrated 
Helmet  and  Display  Sighting  System  (IHADSS),  which  has  been  used  in  the  AH-64  Apache  since  the 
1980’s.  It  is  similar  to  the  JHMCS  in  that  imagery  from  a  CRT  is  projected  onto  a  partially  reflective 
piece  of  glass  in  front  of  the  pilot’s  right  eye.  This  allows  the  pilot  to  view  both  the  outside  world  and  the 
IHADSS  imagery  simultaneously  in  the  right  eye.  Pilots  can  view  symbology  as  well  as  Forward 
Looking  Infrared  (FLIR)  imagery  using  the  IHADSS  (Behar  et  al,  1990). 

The  potential  for  binocular  rivalry  and  significant  visual  discomfort  during  extended  use  of  the 
IHADSS  was  a  major  concern  so  a  number  of  AH-64  pilots  were  interviewed  and  tested  before,  during, 
and  after  use  of  the  IHADSS  (Behar  et  al,  1990).  The  results  of  the  study  indicated  that  a  large 
percentage  of  pilots  reported  at  least  one  visual  complaint  associated  with  flying  an  IHADSS  equipped 
aircraft.  Complaints  consisted  of  visual  discomfort,  headache,  double  vision,  blurred  vision, 
disorientation,  and  afterimages.  Of  pilots  complaining  of  visual  difficulty  the  most  common  problem  was 
blurred  vision  in  right  eye  and/or  headache.  Twenty  percent  of  pilots  reported  afterimages,  particularly 
after  viewing  FLIR  imagery  in  the  HMD  after  long  flights.  About  70%  of  pilots  noted  that  vision 
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occasionally  and  unintentionally  alternated  between  their  left  and  right  eye  either  during  or  after  flight.  A 
few  pilots  also  reported  “browning”  or  “brown  outs”  in  the  right  eye  following  usage  of  the  IHADSS  for 
long  periods  of  time. 

An  analysis  of  the  diopters  settings,  the  focusing  corrections  set  by  the  pilots  while  using  the 
IHADSS,  indicated  that  the  settings  were  incorrectly  adjusted  and  ranged  from  0  to  -5.25  diopters,  with  a 
mean  of  -2.28.  These  incorrect  focus  settings  would  cause  significant  eyestrain  and  discomfort  during 
extended  use.  Behar  et  al  therefore  concluded  that  the  source  of  the  pilots’  visual  complaints  could 
therefore  likely  be  attributed  to  incorrect  focus  adjustment,  which  could  be  corrected  with  proper  training, 
rather  than  the  occurrence  of  binocular  rivalry.  Nevertheless,  the  potential  for  rivalry  was  not  ruled  out. 
The  simultaneous  viewing  of  the  real  world  and  FLIR  imagery,  particularly  at  night,  could  create 
conditions  conducive  to  rivalry  due  to  the  significant  difference  in  brightness,  contrast,  color,  flicker,  and 
shape  of  the  imagery  in  the  two  eyes. 

In  another  study,  it  was  concluded  that  transparent  monocular  HMDs  may  not  be  suitable  for  use 
under  conditions  where  the  background  (OTW)  scene  is  complex  and  dynamic  (Laramee  &  Ware,  2002) 
due  to  the  occurrence  of  binocular  rivalry.  The  drive  to  display  more  information  to  pilots  of  fixed  wing 
aircraft  will  create  similar  conditions.  FLIR  and  other  sensor  imagery  may  find  its  way  into  devices  like 
the  JHMCS.  For  example,  the  HMD  to  be  incorporated  in  the  JSF  is  being  designed  to  display  imagery 
from  various  sensors  in  addition  to  symbology.  It  is  therefore  important  to  determine  the  parameters  of 
the  monocular  HMD  that  minimize  rivalry  and  visual  suppression,  as  well  as  any  other  sources  of  visual 
discomfort,  and  establish  longer  periods  of  effective  use. 

Basic  Research  on  Binocular  Rivalry 

To  establish  a  foundation  for  future  research  on  binocular  rivalry  a  brief  review  of  previous 
research  follows. 

Binocular  rivalry  has  been  shown  to  occur  when  completely  different  images  are  displayed  to  the 
two  eyes  (see  Figure  6).  For  example,  a  grid  of  vertical  bars  shown  to  the  left  eye,  and  a  grid  of 
horizontal  bars  shown  to  the  right  eye  will  produce  rivalry.  Differences  of  luminance  and  color  between 
the  two  eyes  have  been  shown  to  induce  rivalry  (LeVelt,  1965).  Reversed  contrast  imagery  in  the  two 
eyes  will  also  produce  rivalry.  Differences  in  brightness  may  produce  the  appearance  of  a  polished 
surface.  Rivalry  induced  by  color  difference  is  difficult  to  obtain  and  only  arises  in  carefully  controlled 
conditions.  In  those  cases  the  two  different  colors,  presented  separately  to  the  two  eyes  appear  to  fuse 
into  a  color  that  is  the  mixture  of  the  two  separate  colors. 
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Figure  6.  Conditions  producing  binocular  rivalry. 


LeVelt  (1965)  makes  a  distinction  between  two  different  forms  of  rivalry:  true  interactive  rivalry 
and  spurious  rivalry.  Interactive  rivalry  is  the  suppression  of  color,  contour,  or  brightness  in  one  eye  by 
the  stimulation  of  the  other  eye.  Spurious  rivalry  occurs  when  a  homogenous  field  is  presented  to  one  eye 
and  a  contoured  target  is  presented  to  the  second  eye.  The  contoured  target  will  dominate  when  viewed 
centrally,  but  will  fade  when  viewed  peripherally  (despite  the  fact  that  contours  generally  dominate  under 
these  conditions).  During  spurious  rivalry  then,  fixated  targets  will  likely  maintain  visibility  but  targets  in 
the  periphery  may  periodically  fade  from  view.  LeVelt  contends  that  this  peripheral  fading  of  imagery  is 
not  due  to  interactions  between  the  two  eyes  but  rather  spontaneous  fading  of  imagery  referred  to  as 
Troxler’s  effect.  This  effect  can  also  occur  during  binocular  presentation.  True  interactive  rivalry  must 
therefore  involve  more  than  a  homogenous  field  in  one  eye  (i.e.,  contour-contour  rivalry,  color-color 
rivalry,  etc.).  Interactive  rivalry  and  spurious  rivalry  may  occur  simultaneously  however  (LeVelt,  1965). 
Whether  spurious  or  interactive,  portions  of  presented  imagery  may  disappear. 

If  images  of  two  different  luminance  levels  are  presented  to  each  eye  separately  the  resulting 
percept  will  be  intermediate  between  the  two  luminance  levels  and  is  referred  to  as  brightness  averaging. 
If  the  observer  closes  one  eye  to  look  only  at  the  brighter  field,  he/she  perceives  a  brighter  image 
compared  to  the  perceived  brightness  with  both  eyes  open.  This  perceived  decrease  in  brightness  despite 
the  fact  that  the  total  energy  delivered  to  the  two  eyes  is  greater  than  that  of  the  single  brighter  image  seen 
by  one  eye  is  referred  to  as  Fechner’s  paradox.  In  a  series  of  experiments  LeVelt  shows  how  fields  of 
differing  luminance  presented  to  the  two  eyes  are  weighted,  and  further,  shows  the  effect  of  the  presence 
of  contours  in  one  eye  or  the  other  on  perceived  brightness.  LeVelt  concludes  that  the  effect  of  contours 
is  a  local  effect,  concentrated  near  the  contour  (as  opposed  to  affecting  brightness  perception  across  the 
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whole  eye),  and  that  this  mechanism  is  what  leads  to  alternate  fading  between  the  left  and  right  eyes  in  a 
more  complex  scene  with  more  extensive  contours  combined  with  eye  movements.  The  alternation  of 
dichoptically  viewed  stimuli  can  be  described  by  frequency  and  dominance.  Frequency  refers  to  how 
often  the  left  and  right  images  seem  to  alternate  to  the  observer,  while  dominance  refers  to  the  total  time 
that  the  left  and  right  images  are  visible  to  the  observer.  Dominance  and  alternation  have  been  found  to 
be  influenced  by  differences  in  contrast  and  stimulus  size  -  or  what  LeVelt  refers  to  as  amount  of  contour 
per  area. 

Yang,  Rose,  and  Blake  (1992)  note  that  a  variety  of  perceptual  states  may  occur  during  dichoptic 
viewing.  One  state  is  monocular  dominance  where  the  image  presented  to  one  eye  is  entirely  visible  and 
alternating  with  the  complete  image  in  the  other  eye.  A  second  state  is  a  mosaic  consisting  of  portions  of 
each  image  presented  separately  to  the  two  eyes.  A  third  state  is  the  appearance  of  superimposed  images 
that  appear  to  be  at  the  same  depth.  Yang  et  al  refer  to  this  as  superimposition.  A  fourth  state  is  similar 
to  superimposition  except  that  the  two  images  appear  to  be  at  different  depths  and  is  referred  to  as 
transparency.  Yang  et  al  found  that  monocular  dominance  tends  to  occur  when  the  two  competing  images 
are  of  similar,  and  generally  lower,  spatial  frequencies.  Superimposition  was  found  to  occur  when  the 
two  images  were  of  similar  but  higher  spatial  frequency.  Transparency  was  found  to  occur  more  often 
when  the  spatial  frequencies  of  the  two  images  were  significantly  different  from  one  another  and  when 
one  of  the  images  had  a  high  spatial  frequency.  The  spatial  frequency  content  of  the  two  images  therefore 
influence  the  observer’s  resulting  perception. 

It  has  generally  been  assumed  that  binocular  rivalry  arises  due  to  conflicting  input  from  the  two 
eyes.  In  other  words,  the  conflict  occurs  early  in  visual  processing  due  to  two  conflicting  images  from  the 
two  eyes.  However,  several  researchers  (Logothetis,  Leopold,  &  Sheinberg,  1996,  Kovacs,  Papathomas, 
Yang,  &  Feher,  1996)  have  shown  that  rivalry  might  be  thought  of  as  two  “conflicting  stimuli”  rather  than 
“conflicting  eyes”.  In  other  words,  the  conflict  may  occur  late  in  visual  processing,  at  cortical  levels. 
Binocular  rivalry  might  therefore  be  similar  to  other  bistable  stimuli  (such  as  the  figure  or  ground 
alternation).  Logothetis  et  al  (1996)  alternated  rivalrous  stimuli  and  compared  dominance  in  this 
condition  with  typical  static  binocular  stimuli.  They  found  that  a  dominant  stimulus  could  remain 
dominant  even  though  it  had  switched  from  the  “dominant  eye”  to  the  “non-dominant”  eye.  Kovacs  et  al 
(1996)  compared  typical  static  rivalrous  stimuli  with  stimuli  that  had  been  broken  into  sections  and 
swapped  between  left  and  right  images.  They  found  that  observers  effectively  reassembled  the  broken 
images  presented  separately  to  the  two  eyes.  Furthermore,  frequency  of  rivalry  alternation  remained 
equal  between  the  two  conditions.  This  could  not  occur  if  rivalry  was  simply  the  result  of  the  suppression 
of  one  eye’s  input  over  the  other.  These  findings  imply  that  imagery  presented  separately  to  the  two  eyes 
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may  be  less  likely  to  provoke  rivalry  if  they  can  be  thought  of  as  forming  a  complete  percept,  for  example 
if  one  half  of  a  face  were  presented  to  the  left  eye,  and  the  other  half  to  the  right  eye,  observers  may 
simply  meld  the  two  without  significant  alternation,  or  competition  between  the  two  images. 

In  support  of  this,  it  has  been  shown  that  neurons  in  the  brain  early  in  visual  processing  do  not 
exhibit  responses  indicative  of  rivalry  (alternating  levels  of  activity).  Recordings  of  the  activity  of 
neurons  farther  downstream  (in  the  cortex),  however,  do  show  evidence  of  alternating  patterns  when  two 
different  images  are  presented  to  the  two  eyes  (Blake  &  Logothetis,  2002). 

Ocular  Dominance 

Related  to  binocular  rivalry  is  the  concept  of  ocular  dominance.  This  refers  to  a  person’s 
preference  for  input  from  one  eye  or  the  other.  Ocular  dominance  has  been  tested  in  numerous  ways. 
Porac  and  Coren  (1976)  identified  eight  tests  of  dominance,  which  include  sighting  tests,  unconscious 
sighting  tests,  binocular  rivalry  tests,  acuity  tests,  motoric  efficiency  tests,  clarity  tests,  perceptual 
efficiency  tests,  and  measurements  of  innervation  density.  Coren  and  Kaplan  (1973)  performed  a  factor 
analysis  on  data  from  a  battery  of  13  tests  for  ocular  dominance.  The  results  indicate  three  factors  to  be 
most  important:  sighting,  sensory,  and  acuity  dominance. 

Sighting  dominance:  These  tests  seek  to  observe  which  eye  is  consistently  favored  during 
monocular  viewing  tasks  (such  as  when  looking  through  a  telescope).  Many  variations  of  this  test  have 
been  proposed  (Crider,  1944;  Coren  &  Kaplan,  1973),  some  of  which  are  designed  to  negate  any  effect  of 
hand  dominance  by  requiring  participants  to  hold  the  sighting  apparatus  with  both  hands  (unconscious 
sighting  tests.  Miles,  1929,  1930;  Gronwall  &  Sampson,  1971).  Sighting  tests  are  simple  to  perform  -  a 
test  that  your  optometrist  may  use  during  an  eye  exam  simply  involves  forming  a  triangle  with  two  hands 
and  looking  through  the  center  of  the  triangle  at  a  target  on  the  wall.  The  target  is  then  viewed  alternately 
with  one  eye,  then  the  other  while  the  opposite  eye  is  closed.  The  eye  that  is  most  in  line  with  the  target 
is  identified  as  the  dominant  eye.  When  viewing  the  target  through  the  triangle,  if  the  dominant  eye  is 
closed  to  view  the  target  with  the  other  eye,  the  target  will  appear  to  shift  to  one  side.  For  example,  if  a 
person  is  left  eye  dominant  and  then  closes  the  left  eye  to  view  the  target  centered  in  the  triangle  with  the 
right  eye,  the  target  will  appear  to  shift  to  the  right.  This  test  is  similar  to  the  “hole  test”  described  by 
Coren  and  Caplan  (1973). 

Due  to  their  relative  simplicity,  sighting  tests  seem  to  be  the  most  common  test  for  ocular 
dominance.  In  addition,  the  results  for  sighting  dominance,  compared  to  other  forms  of  dominance, 
appear  to  be  the  most  reliable  (Porac  &  Coren,  1976).  Right-eyed  sighters  have  shown  more  consistency 
in  their  preference  than  left-eyed  sighters  (Friedlander,  1971;  Porac  &  Coren,  1975).  Not  only  are  right¬ 
eyed  sighters  more  consistent,  but  they  seem  to  be  more  numerous.  Estimates  from  a  variety  of  studies 
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indicate  that  about  60  -  70  %  of  the  population  are  right-eyed  sighters  while  30  -  40  %  tend  to  be  left¬ 
eyed  sighters  (Porac  &  Coren,  1976). 

Sensory  dominance  tests  seek  to  observe  which  eye’s  input  is  favored  longest  during  binocular 
rivalry  tasks.  This  type  of  dominance  has  received  far  less  attention  than  sighting  dominance.  However, 
like  sighting  tests,  binocular  rivalry  tests  have  shown  a  preference  for  the  right  eye  (Cohen,  1952;  Coren 
&  Kaplan,  1973,  Porac  &  Coren,  1976).  Some  of  the  difficulties  in  this  approach  are  that  the  results  may 
vary  depending  on  the  size,  color,  contrast  of  the  stimuli  (Mapp,  Ono,  &  Barbeito,  2003).  One  method  of 
testing  sensory  dominance  was  developed  by  Washburn,  Faison,  and  Scott  (1934).  Their  method  involves 
red  and  blue  cards.  Observers  view  the  cards  through  a  stereoscope,  such  that  one  card  is  visible  to  the 
right  eye,  and  the  other  card  is  visible  only  to  the  other  eye.  The  observers  are  then  asked  to  indicate 
which  color  they  see  and  when  the  color  changes  (i.e.,  red,  blue,  or  purple).  The  experimenter  simply 
records  the  time  for  which  each  color  is  dominant.  LeVelt  (1965)  describes  a  similar  method  in  which 
equibrightness  curves  are  computed  based  on  brightness  matches  between  a  test  field  presented  to  one  eye 
and  a  separate  field,  which  the  observer  can  adjust,  presented  to  the  other  eye.  The  slope  of  the  resulting 
function  indicates  eye  dominance.  Behar  et  al  (1990)  used  a  relatively  simple  technique  to  assess  eye 
dominance  in  which  an  observer  viewed  moving  bars  displayed  on  a  monitor.  A  prism  was  placed  in 
front  of  one  eye  so  the  direction  of  motion  was  different  for  that  eye.  The  perceived  direction  of  motion 
was  recorded  over  time  thus  establishing  eye  dominance. 

Acuity  measures  of  dominance  simply  involve  testing  each  eye  separately  for  acuity.  The  eye 
with  the  best  resolving  power  is  identified  as  the  dominant  eye. 

Sighting  tests  of  dominance  are  clearly  the  easiest  to  perform  and  are  the  most  frequently 
administered.  The  relative  prevalence  of  right-eyed  sighters  is  evident  and  current  monocular  HMDs 
such  as  the  JHMCS  are  typically  mounted  over  the  right  eye.  However,  the  various  tests  for  ocular 
dominance  are  generally  not  correlated  (Mapp,  Ono,  &  Barbeito,  2003).  A  sighting  test  may  indicate 
right  eye  dominance  for  example,  while  a  sensory  dominance  test  indicates  that  either  the  left  eye  is 
dominant  or  neither  eye  is  dominant.  Dominance  identified  using  sensory  dominance  tests  may  even  vary 
depending  on  the  type  of  stimuli  used  (e.g.,  dominance  evoked  with  contours  may  be  different  from 
dominance  evoked  by  color).  It  is  therefore  unclear  at  this  point  what  significance  measures  of  eye 
dominance  have  or  whether  they  can  predict  anything  about  performance  during  the  use  of  HMDs. 

Out-the-Window  (OTW)  Viewing 

We  consider  here  the  situation  of  using  binocular  HMDs  for  providing  the  OTW  simulated 
imagery  for  flight  training  purposes.  This  simulation  can  involve  either  a  real  HUD  and  cockpit  seen 
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through  a  semi-transparent  HMD  depicting  an  out-the-window  scene,  or  the  HMD  imagery  simulating 
both  a  virtual  HUD  and  an  OTW  scene.  In  the  latter  case,  an  occluding  HMD  may  also  work,  provided 
there  is  ample  look-under  clearance  to  view  cockpit  instruments. 

When  simulating  OTW  viewing,  one  can  use  either  a  biocular  HMD  or  a  binocular  HMD. 
Biocular  HMDs  present  the  same  imagery  to  both  eyes  of  a  user.  Depth  and  distance  would  be  conveyed 
by  monocular  cues  such  as  perspective  and  motion  parallax  presented  to  both  eyes  with  zero  binocular 
parallax.  Binocular  HMDs  present  similar  imagery  to  the  two  eyes  of  the  user  with  binocular  parallax 
between  the  views,  which  is  achieved  by  presenting  each  eye's  view  from  a  different  eye  point.  Depth 
and  distance  would  be  conveyed  by  monocular  cues  together  with  binocular  parallax. 

Ideally  an  HMD  designed  for  display  of  simulated  imagery  for  OTW  viewing  for  training 
applications  would  be  lightweight,  un-tethered,  have  a  large  field  of  view  equal  to  the  field  of  view  of  the 
human  visual  system,  eye-limited  spatial  resolution,  and  full  binocular  overlap  for  depth  perception.  In 
reality,  however,  each  of  these  factors  will  trade  off  with  one  another,  and  the  combination  of  all  these 
characteristics  is  currently  unachievable.  For  example,  current  light  modulators  are  generally  limited  to 
about  1280  x  1024  pixels.  If  eye-limited  resolution  is  a  driving  factor,  the  field  of  view  is  therefore 
necessarily  limited  to  approximately  20  degrees  per  eye  (i.e.,  1  arcmin  per  pixel).  If  field  of  view  is  the 
driving  factor  the  weight  and  size  of  the  optics  becomes  an  important  consideration.  As  field  of  view 
increases,  the  weight  and  size  of  the  optics  required  increases  rapidly,  while  resolution  decreases.  Melzer 
and  Moffitt  (1997)  and  Velger  (1998)  discuss  these  important  design  issues  in  greater  depth.  Reducing 
the  amount  of  cabling  used  to  provide  imagery  and  power  to  the  HMD  is  also  very  important.  For 
example,  a  significant  complaint  in  an  evaluation  of  the  CAE  advanced  fiber-optic  helmet-mounted 
display  (FOHMD)  was  that  the  thick  fiber  optic  bundles  inhibited  head  movement  (Brown  et  ah,  1996). 

It  is  not  currently  possible  to  remove  the  cabling  entirely  but  it  may  be  possible  to  transmit  imagery  via  a 
thin,  lightweight  fiber-optic  cable  to  reduce  factors  inhibiting  head  movements  as  much  as  possible. 
Overcoming  these  difficulties  pose  significant  engineering  challenges.  However,  we  will  once  again 
focus  here  on  the  perceptual  issues  likely  to  be  encountered  in  OTW  HMD  applications. 

Perceptual  Issues  in  OTW  HMD  Applications 

Field  of  View 

The  field  of  view  of  an  HMD  is  an  important  consideration  when  designing  and  implementing 
any  given  HMD  system.  Field  of  view  is  important  for  the  obvious  reason  that,  other  factors  being 
constant,  a  larger  field  of  view  will  better  simulate  natural  viewing.  The  visual  field  of  the  human  visual 
system  extends  200  deg  horizontal  by  130  deg  vertical,  with  the  central  120  deg  being  the  area  of 
binocular  overlap  (Velger,  1998,  pp.  50).  Wide  fields  of  view  can  be  created  with  relatively  large  optics 
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(and  therefore  increasing  the  weight  of  the  system)  or  smaller  eye  relief  (Velger,  1998,  pp.  64-65). 
However,  increasing  field  of  view  typically  lowers  display  resolution  because  the  same  pixels  are  mapped 
to  a  larger  display  area  as  field  of  view  is  increased,  which  therefore  decreases  resolution  (Fisher,  1994). 

Field  of  view  may  affect  perceived  size  and  distance  as  well  as  heading  performance.  These 
effects  include  the  underestimation  of  size  and  distance  in  the  display  and  impaired  visual  cognition 
(Osgood  &  Wells,  1991).  Brickner  and  Foyle  (1990)  report  that  heading  control  in  a  flight  simulator  was 
impaired  when  the  field  of  view  was  25  deg  relative  to  a  larger  field  of  view  of  55  deg. 

Pilots  testing  HMDs  for  OTW  viewing  applications  invariably  prefer  large  fields  of  view  and  one 
would  generally  expect  that  performance  on  a  wide  variety  of  tasks  should  be  negatively  affected  with  a 
restricted  field  of  view.  However,  research  investigating  pilot  performance  on  various  tasks  has  yielded 
mixed  results  and  clear  recommendations  on  minimum  field  of  view  for  flight  training  applications  have 
not  been  established.  To  shed  some  light  on  the  effects  of  field  of  view  in  HMD  OTW  applications  a 
review  of  the  published  vision  research  literature  on  this  topic  follows. 

The  visual  system  is  anatomically  subdivided  into  different  visual  pathways  -  the  parvocellular 
and  magnocellular  pathways  (Livingstone  &  Hubei,  1988;  Schiller  et  ah,  1990).  The  parvocellular 
pathway  has  connections  from  the  central  retina  and  project  mainly  to  areas  in  the  visual  cortex  (located 
in  the  occipital  lobe  toward  the  back  of  the  head)  that  make  up  the  cortical  ventral  stream.  Areas  in  the 
ventral  stream  functionally  analyze  spatial  pattern  information.  Cells  in  these  cortical  areas  have  a 
sustained  and  sluggish  response  and  poor  temporal  acuity.  Many  cells  in  these  areas  respond  to  color 
information  and  they  have  fine  spatial  acuity  for  functionally  analyzing  fine  detail  and  high  spatial 
frequency  information.  In  general,  the  parvocellular  pathway  is  thought  to  be  involved  in  the  functional 
analysis  of  spatial  pattern  information  for  the  purpose  of  identifying  objects. 

The  magnocellular  pathway,  in  contrast,  has  connections  from  the  central  and  peripheral  retina, 
and  projects  mainly  to  areas  in  the  visual  cortex  that  make  up  the  cortical  dorsal  stream.  Areas  in  the 
dorsal  stream  functionally  analyze  optic  flow  information  for  heading  control  (Peuskens,  Sunaert, 

Dupont,  Van  Hecke  &  Orban,  2001),  biological  motion  (Grossman,  Donnelly,  Price,  Morgan,  Pickens, 
Neighbor  &  Blake,  2000;  Grossman  &  Blake,  2001),  and  they  integrate  vision  with  action  (e.g.,  Yabuta, 
Sawatari  &  Callaway,  2001;  Ts'o  &  Roe,  1995;  Van  Essen  &  DeYoe,  1995).  Cells  in  these  cortical  areas 
have  a  transient  response  and  high  temporal  acuity.  Cells  in  these  areas  do  not  respond  to  color 
information  and  they  possess  poor  spatial  acuity;  they  functionally  analyze  coarse  detail  and  moderate  or 
low  spatial  frequency  information.  In  general,  the  magnocellular  pathway  is  thought  to  be  involved  in  the 
functional  analysis  of  motion  information  for  the  purpose  of  locating  objects,  determining  spatial 
relations,  controlling  heading  during  locomotion,  and  processing  spatial  orientation. 
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In  a  recent  book,  Milner  and  Goodale  (1996)  make  a  further  distinction  between  the  two 
pathways.  They  claim  that  the  parvocellular  pathway  is  involved  only  in  the  representation  of  visual 
experience  while  the  magnocellular  pathway  is  involved  in  the  completely  separate  process  of  priming 
action  and  motor  control.  This  view  implies  that  any  information  gleaned  from  research  on  visual 
judgments  and  reports  would  not  necessarily  generalize  to  actual  sensory-motor  control  tasks.  Their 
claim  is  quite  different  from  the  typical  view  of  the  two  pathways,  namely  that  the  parvocellular  pathway 
is  involved  in  the  functional  analysis  of  spatial  pattern  information  while  the  magnocellular  pathway  is 
involved  in  the  functional  analysis  of  motion  information.  In  support  of  this  notion,  Richman,  Stanfield, 
and  Dyre  (1998)  found  that  heading  performance  differed  for  active  control  versus  passive  viewing.  They 
also  found  that  heading  control  was  differentially  affected  by  field  of  view  for  active  control  versus 
passive  viewing.  With  a  large  field  of  view  (90  degrees),  observers  performed  a  heading  task  with  less 
error  using  active  control.  For  a  small  field  of  view  (30  degrees)  however,  errors  on  the  heading  task 
were  larger  for  the  active  control  task  compared  to  the  passive  viewing  task. 

Turning  back  to  the  problem  of  employing  HMDs  with  a  restricted  field  of  view,  using  a  HMD 
with  a  restricted  field  of  view  would  mean  that  stimulation  would  occur  mainly  in  the  central  portion  of 
the  retinae  of  the  two  eyes.  Although  eye  movements  are  usually  possible  with  a  HMD,  it  is  likely  that 
the  user  would  spend  much  of  the  time  looking  in  the  forward  direction.  This,  in  turn,  would  mean  that 
the  parvocellular  pathway,  rather  than  the  magnocellular  pathway,  would  be  predominantly  activated. 
Under  such  conditions,  an  observer's  ability  to  control  heading  during  locomotion  and  to  process  spatial 
orientation  might  be  compromised,  as  might  be  the  sense  of  immersion. 

For  example,  Allison,  Howard,  and  Zacher  (1999)  investigated  the  effect  of  the  size  of  the  field- 
of-view  on  roll  vection  and  illusory  self-tilt  in  a  tumbling  room.  These  authors  found  that  complete  360- 
degree  body  rotation  (tumbling)  and  increased  speed  of  self-rotation  was  perceived  by  most  observers 
under  a  full  field  of  view  condition.  For  smaller  fields  of  view,  tilt  or  partial  tumbling  and  slower  speeds 
of  self-rotation  were  perceived.  When  the  field  of  view  was  50  degrees,  only  about  one-half  of  all 
observers  reported  tumbling  at  an  intermediate  speed.  This  suggests  that  a  field  of  view  of  at  least  50 
degrees  is  needed  for  some  sense  of  immersion,  and  that  a  very  large  field  of  view  would  be  needed  for  a 
complete  sense  of  immersion. 

Similarly,  Duh,  Lin,  Kenyon,  Parker,  and  Furness  (2001)  found  that  research  participants’  ability 
to  balance  was  increasingly  impaired  as  field  of  view  was  increased.  In  this  experiment  participants  stood 
inside  a  3  foot  dome  while  a  variety  of  scenes  were  projected  and  oscillated.  Pressure  plates  were  used  to 
monitor  center  of  balance.  Fields  of  view  from  30  degrees  to  180  degrees  were  tested.  Surprisingly, 
postural  stability  did  not  plateau  even  at  the  largest  field  of  view  tested  (180  degrees).  This  suggests  that 
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a  sense  of  total  immersion  would  require  a  very  large  field  of  view,  greater  than  180  degrees.  Subjective 
ratings  of  difficulty  in  maintaining  postural  stability  did  begin  to  plateau  at  approximately  120  degrees 
however.  This  may  provide  a  more  attainable  goal  for  HMD  designers  while  still  providing  a  sufficient 
feeling  of  immersion. 

Tasks  requiring  accurate  sense  of  self-motion,  or  the  positioning  of  an  aircraft  relative  to  other 
aircraft,  such  as  during  formation  flying,  may  also  require  large  fields  of  view.  For  example,  Weikhorst 
and  Vacarro  (1988)  found  that  maneuvers  such  as  the  aileron  and  barrel  roll  were  performed  significantly 
better  by  pilots  in  a  simulator  with  a  wide  field  of  view  (300  degrees  horizontally),  compared  to  limited 
fields  of  view  (144  degrees  and  36  degrees  horizontally).  They  also  found  that  accuracy  in  dropping 
bombs,  in  terms  of  meters  to  the  target,  improved  when  field  of  view  was  increased.  These  authors 
further  noted  that  the  flight  paths  of  pilots  dropping  bombs  in  the  limited  field  of  view  conditions  were 
significantly  different  from  flight  paths  in  the  wide  field  of  view  condition.  Significantly  reducing  field 
of  view  might  therefore  have  serious  training  implications.  In  another  study  examining  pilot  performance 
in  conducting  realistic  tactical  maneuvers  while  field  of  view  was  varied,  Dixon,  Krueger,  Rojas,  and 
Hubbard  (1989)  found  that  increased  field  of  view  reduced  the  number  of  trials  to  train  a  low-level  flight 
maneuver  as  well  as  a  30-degree  manual  dive-bombing  task.  Based  on  the  results,  they  recommended  a 
minimum  instantaneous  field  of  view  of  160  x  80  degrees  for  flight  simulation  training  applications. 

Experiments  examining  field  of  view  (Kruk  &  Runnings,  1989)  with  the  CAE  fiber-optic  helmet 
mounted  display  (FOHMD)  did  not  find  statistically  reliable  differences  in  performance  during  various 
low  altitude  flight  tasks  as  field  of  view  was  varied  (127,  107,  and  87  degrees).  However,  the  pilots  all 
indicated  that  workload  while  performing  maneuvers  in  the  limited  field  of  view  conditions  was 
substantially  higher.  A  similar  finding  was  noted  for  high  altitude  air-to-air  combat  maneuvers. 
Significant  differences  in  maneuvering  time  were  found  for  formation  flying  as  field  of  view  was 
decreased.  Pilots  in  this  study  found  it  very  difficult  to  maneuver  within  6  feet  wingtip  to  wingtip  on  the 
fingertip  formation,  and  substantial  pilot  induced  oscillation  occurred  in  the  reduced  field  of  view 
conditions.  The  authors  attributed  this  to  lack  of  motion  cuing,  as  well  as  the  reduced  horizon  reference 
for  the  reduced  field  of  view  conditions. 

Other  studies  examining  field  of  view  however  often  show  a  plateau  at  approximately  40  degrees. 
For  example,  in  a  study  involving  a  simulated  nighttime  attack,  Osgood  and  Wells  (1991)  examined  size 
of  the  field  of  view  for  an  aircraft-fixed  sensor  whose  image  was  seen  on  a  HUD  and  for  a  head-steered 
sensor  seen  on  a  helmet-mounted  display.  With  the  head-steered  HMD  system,  and  for  larger  fields  of 
view,  observers  acquired  the  targets  quicker  up  to  a  field-of-view  of  40  degrees,  beyond  which 
performance  did  not  significantly  change.  Kenyon  and  Kneller  (1993)  investigated  the  effect  of  field  of 
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view  on  a  visual  tracking  task  and  found  that  near-asymptotic  performance  was  obtained  with  a  field  of 
view  of  only  40  degrees. 

Generally,  the  difference  between  those  experiments  that  find  that  a  large  field  of  view  is 
beneficial  and  those  that  find  that  limited  fields  of  view  (less  than  60  degrees)  are  sufficient  is  the  type  of 
task  that  is  examined.  Based  on  the  hypothesis  proposed  above,  performance  of  tasks  primarily  engaging 
the  magnocellular  visual  processing  system  will  benefit  from  a  very  large  field  of  view  (greater  than  60 
degrees),  while  those  tasks  primarily  engaging  the  parvocellular  visual  processing  system  can  be 
performed  accurately  with  a  limited  field  of  view  (less  than  60  degrees).  For  example,  formation  flying, 
and  basic  combat  maneuvers,  such  as  barrel  rolls,  tasks  that,  in  short,  require  accurate  perception  of  self- 
motion  (or  simulated  self-motion  in  this  case),  benefit  from  a  large  field  of  view.  Targeting  tasks, 
tracking  tasks,  and  object  recognition  tasks,  in  contrast,  are  performed  accurately  with  field  of  view  as 
small  as  40  degrees.  Additional  research  should  be  performed  with  visual  tasks  chosen  using  this 
framework  to  establish  a  field  of  view  for  HMD  applications  that  can  support  adequate  performance  for 
perception  of  self-motion  while  still  maintaining  feasible  optical  size,  weight,  and  resolution.  A  concept 
that  might  be  considered,  for  example,  is  a  small  field  of  view,  high-resolution  central  area,  with  a  low- 
resolution  wide  field  of  view  surrounding  area.  This  might  provide  the  high-resolution  detail  necessary  to 
support  target  identification  without  reducing  the  perception  of  optic  flow  or  other  cues  necessary  for  the 
perception  of  self-motion. 

Interocular  differences 

With  binocular  HMDs,  the  content  of  the  imagery  and  scenes  projected  to  the  two  eyes  is  usually 
different  only  in  a  way  consistent  with  binocular  parallax  and  rivalry  usually  does  not  ensue.  When  the 
content  of  the  synthetic  scenes  delivered  to  the  two  eyes  is  the  same  or  very  similar,  binocular  fusion 
results.  However,  the  content  of  imagery  shown  to  each  eye  could  vary  considerably  in  practice.  For 
example,  designers  could  present  symbology  to  one  eye  while  both  eyes  view  the  OTW  scene,  in  a 
manner  similar  to  the  JHMCS.  As  another  example,  FLIR  imagery  could  be  displayed  to  one  eye  while 
the  other  eye  views  the  OTW  scene.  If  these  types  of  situations  can  be  expected  it  will  be  important  to 
determine  characteristics  of  binocularly  presented  imagery  that  reduce  the  occurrence  of  rivalry. 
Additionally,  slight  interocular  differences  in  certain  image  characteristics,  which  still  allow  fusion  to 
occur,  can  nonetheless  cause  unwanted  perceptual  effects.  For  example,  slight  differences  in  line 
orientation  between  images  delivered  to  the  two  eyes  will  create  the  perception  of  a  line  tilted  through  the 
Z-axis,  as  well  as  a  cyclotorsion  response  (Pierce,  Arrington,  &  Moreno,  1999).  Over  a  period  of  time, 
this  could  create  muscle  fatigue  and  eye  strain.  Additionally,  slight  differences  in  the  size  of  images  can 
create  the  perception  of  an  object  rotated  about  the  vertical  axis. 
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With  regard  to  visual  tolerance  for  interocular  differences  in  simulation  using  a  binocular  HMD, 
Rash,  Mozo,  McEntire,  and  Licina  (1996)  suggest  that  interocular  differences  in  luminance  of  up  to  30%, 
a  rotational  difference  of  up  to  10  arcmin,  horizontal  or  vertical  differences  in  image  size  of  up  to  1.5%, 
and  deviation  between  centers  of  the  two  displays  of  0.18  prism  diopters  (6.12  arcmin),  all  should  be 
acceptable. 

Convergent  and  divergent  partial  overlap  displays  (see  Figure  7)  can  produce  perceptual  conflicts 
such  as  tuning,  which  is  a  form  of  rivalry  in  which  there  is  a  subjective  darkening  of  the  monocular 
regions  near  the  border  with  the  binocular  overlap  region  (Velger,  1998).  Luning  can  be  minimized  by 
reducing  the  luminance  near  the  edges  of  the  binocular  region  (Velger,  1998,  pp.  56;  Melzer  &  Moffitt, 
1991;  Klymenko  et  ah,  1994;  Grigby  &  Tsou,  1994).  Luning  may  be  related  to  interocular  suppression 
affecting  unpaired  monocular  images  (called  half-occlusions)  which  do  not  normally  occur  with  occlusion 
in  the  real  world  (Nakayama  &  Shimojo,  1990).  With  binocular  viewing,  unpaired  images  in  one  eye 
undergo  either  visual  suppression  or  are  perceived  with  depth  depending  upon  whether  the  stimulus 
arrangement  has  ecological  validity  in  terms  of  occlusion.  It  is  therefore  important  to  determine  whether 
problems  with  convergent  versus  divergent  binocular  displays  are  related  to  stimulus  conditions  that 
mimic  occlusion  (Nakayama  &  Shimojo,  1990);  and  whether  one  type  of  display  may  be  better  than  the 
other  in  terms  of  less  visual  suppression  in  the  non-overlapping  portions  of  the  field  of  view  (see  Rash, 
Mozo,  McEntire,  &  Licina,  1996  for  more  discussion  of  luning).  Klymenko  et  al  (2001)  found  that 
convergent  overlap  reduced  reaction  times  in  a  visual  search  task  compared  to  divergent  partial  overlap. 
However,  they  also  found  that  reaction  times  were  significantly  lower  in  the  full  overlap  condition 
compared  to  either  the  divergent  or  convergent  partial  overlap  conditions. 
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Figure  7.  Binocular  overlap  conditions. 

It  will  be  important  to  determine  how  much  binocular  overlap  is  needed  and  assess  its  importance 
relative  to  field-of-view  (field-of-view  would  necessarily  have  to  be  reduced  to  increase  binocular 
overlap).  For  tasks  where  depth  perception  is  particularly  important,  (e.g.,  a  helicopter  landing 
simulation),  greater  binocular  overlap  may  be  required  and  the  subsequent  reduction  in  field-of-view 
justified.  Kruk  and  Longridge  (1984)  found  that  binocular  presentation  improved  target  detection  for 
observers  using  the  FOHMD  compared  to  monocular  presentation.  However,  they  also  found  that  target 
detection  was  degraded  for  targets  located  in  a  5 -degree  lateral  edge  of  the  binocular  overlap  regions  of 
the  display  (for  both  a  25  and  45  degree  overlap).  Because  a  greater  percentage  of  the  central  viewing 
area  would  be  subject  to  this  reduced  detection  level  for  the  smaller  25  degree  overlap  Kruk  and 
Longridge  recommended  that  an  overlap  greater  than  25  degrees  for  subsequent  HMD  systems.  Grigsby 
and  Tsou  (1994)  state  that  binocular  processing  breaks  down  outside  a  40-degree  central  visual  region,  so 
a  central  binocular  overlap  region  of  40  degrees  may  be  sufficient  for  many  tasks  if  the  observer  is 
generally  looking  at  the  center  of  the  displays.  Furthermore,  a  40-degree  area  overlap  ensures  that,  for  all 
but  the  largest  eye  movements,  the  fovea  will  remain  inside  the  overlap  area.  Continuously  crossing  that 
boundary,  and  breaking  and  re-establishing  binocular  fusion,  would  likely  result  in  fatigue  during 
extended  use  (Grigsby  &  Tsou,  1990). 

Although  tuning  has  been  shown  to  affect  performance  for  relatively  simple  static  imagery  it  may 
be  that  the  effect  is  less  noticeable  for  more  complex  dynamic  imagery.  The  complexity  involved  with 
designing  a  device  with  adjustable  binocular  overlap  and  vergence  angle  makes  research  into  this  issue 
difficult.  However,  additional  research  into  this  issue  will  be  important  to  ensure  important  information  is 
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not  missed  by  an  observer  and  how  adjustments  to  combat  tuning  may  affect  other  factors  such  as  field  of 
view  and  eyestrain  (see  accommodation/vergence  section  below). 

Accommodation/Vergence 

For  OTW  viewing  simulations,  the  display  brightness  and  contrast  should  be  sufficient  for 
simulating  the  viewing  of  the  outside  world.  The  display  brightness  and  contrast  should  be  sufficient  so 
as  to  minimize  the  tendency  for  accommodation  to  drift  toward  dark  focus  and  at  the  same  time  produce  a 
relatively  large  depth  of  focus  so  that  images  appear  sharp  and  in  focus  despite  the  tendency  to  verge  in  a 
depth  plane  different  from  the  display  screen. 

When  simulating  OTW  viewing  using  a  binocular  HMD,  the  stimulus  to  accommodation  would 
be  the  imagery  of  the  HMD.  However,  when  the  user  changes  his  or  her  vergence  angle  to  view 
binocularly  a  virtual  target  in  the  synthetic  scene,  the  vergence  angle  can  be  mismatched  relative  to 
accommodative  demand.  Recall  that  there  are  three  possible  effects  of  accommodation- vergence 
mismatch:  (1)  eye  strain  or  visual  discomfort;  (2)  blurring  of  imagery  (depending  upon  stimulus 
brightness  and  contrast);  and  (3)  misperceived  size,  depth,  speed,  and/or  distance  via  the  operation  of 
perceptual  constancies  (Patterson  &  Martin,  1992).  For  example,  Edgar,  Pope,  and  Craig  (1993)  reported 
that  observers  tended  to  misaccommodate  to  virtual  displays  and  may  misperceive  size  and  distance  of 
objects,  and  Ellis,  Bucher,  and  Menges  (1995)  reported  that  errors  of  judged  depth  of  virtual  objects  are 
associated  with  variation  in  binocular  vergence. 

Lampton,  Gildea,  McDonald,  and  Kolasinski  (1996)  compared  performance  on  several  tasks 
while  users  viewed  imagery  through  several  different  display  types.  One  of  the  tasks  they  used  was  a 
distance  estimation  task.  Distance  estimates  for  a  stationary  object  located  40  feet  from  the  observer  were 
compared  for  a  binocular  HMD,  a  standard  computer  monitor,  and  a  boom  mounted  display.  Real  world 
performance  was  also  assessed.  Median  responses  for  estimates  while  using  the  HMD  were  similar  to  the 
other  conditions.  However,  the  range  of  responses  while  using  the  HMD  was  substantially  larger 
compared  to  the  other  display  types  and  to  the  real  world  condition.  Responses  ranged  from  6  feet  to  300 
feet.  These  results  indicate  that  individual  differences  in  distance  scaling  may  indeed  be  substantial  while 
using  some  HMDs.  Judgments  of  size,  speed,  and  depth  may  therefore  also  be  affected. 

A  variety  of  configurations  can  be  adopted  for  binocular  HMDs.  The  vergence  angle  can  be  set 
so  that:  (a)  the  observer’s  eyes  look  straight  ahead,  (b)  vergence  angle  is  equal  in  depth  relative  to 
accommodative  demand,  or  (c)  vergence  angle  is  set  such  that  gaze  angle  overlaps  (i.e.,  vergence 
condition  similar  to  that  of  looking  at  a  very  near  object)  while  accommodation  may  be  at  a  greater 
distance.  Figure  8  illustrates  these  conditions.  Ideally,  the  configuration  should  be  set  in  order  to 
minimize  eyestrain.  Shibata  (2002)  found  that  eyestrain  was  least  when  vergence  angle  and 
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accommodative  demand  were  matched  in  terms  of  distance.  However,  compromises  may  again  need  to 
be  made  in  establishing  accommodative  and  vergence  configurations.  Recall  that  Klymenko  et  al.  (2001) 
found  that  a  convergent  configuration  reduced  the  effects  of  luning  to  some  extent  in  a  visual  search  task. 
Furthermore,  it  may  also  be  useful  to  consider  dark  accommodation  when  selecting  focal  length  of  the 
HMD  optics. 
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Figure  8.  Accommodation  and  vergence  conditions. 


Head  tracking  lag 

There  are  two  perceptual  frames  of  reference  relevant  to  the  discussion  of  HMDs:  (1) 
environment-centered  frame  of  reference  (where  the  projection  system  is  linked  to  the  external  world  in 
terms  of  ambient  light  reflected  off  natural  objects  and  surfaces);  and  (2)  head-centered  frame  of 
reference  (where  the  projection  system  is  linked  to  the  observer’s  head). 

With  an  environment-centered  frame  of  reference  (i.e.,  viewing  the  world  naturally),  changes  in 
eye  or  head  position  alter  the  pattern  of  retinal  stimulation.  With  a  head-centered  perceptual  frame  of 
reference  (i.e.,  viewing  scenes  projected  on  a  helmet-mounted  display),  changes  in  eye  position  alter 
retinal  stimulation  but  changes  in  head  position  do  not.  Changes  in  retinal  stimulation  that  naturally 
accompany  changes  in  head  position  must  be  simulated  by  computer  and  projected  onto  the  HMD.  For 
example,  in  the  case  of  OTW  viewing,  the  image  on  the  display  would  be  shifted  with  the  same 
magnitude  and  in  a  direction  opposite  to  the  head  motion.  The  shift  of  the  visual  scene  would  occur  in  a 
direction  opposite  to  the  head  motion  to  mimic  the  shift  in  imagery  that  would  occur  within  an 
environment-centered  frame  of  reference. 
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Involuntary  head  movements  occur  when  mission  training  or  rehearsal  occurs  within  an 
environment  involving  acceleration  and/or  vibration.  In  this  case,  involuntary  head  movements  elicit  a 
reflexive  eye  movement  called  the  vestibulo-ocular  reflex  (VOR).  The  VOR  entails  involuntary  eye 
movements  which  compensate  for  (i.e.,  are  in  a  direction  opposite  to)  the  involuntary  head  movements  so 
that  fixation  is  maintained  and  the  image  is  stabilized  on  the  retina.  For  instance,  if  an  individual's  head 
vibrates  upwards,  the  retinal  image  would  slip  downward,  and  the  VOR  would  elicit  a  compensatory 
downward  eye  movement  to  maintain  fixation  on  the  object  during  the  head  movement.  In  an 
environment-centered  frame  of  reference  (real  world),  such  a  compensatory  eye  movement  system  is 
important  for  maintaining  fixation  on  objects  during  locomotion  (Benson  &  Barnes,  1978;  Velger,  1998, 
pp.  207).  Significant  vibration  would  not  be  expected  in  most  flight  training  environments,  however,  in 
shipboard  training  situations,  such  as  on  an  aircraft  carrier,  involuntary  head  movements  might  be  more  of 
a  concern. 

With  HMDs,  which  involve  a  head-centered  frame  of  reference,  compensatory  eye  movements 
(VOR)  disrupt  vision  because,  during  the  involuntary  head  movement  produced  by  vibration,  the  retinal 
image  moves  with  the  head.  The  compensatory  eye  movements  are  inappropriate  because  they  occur  in  a 
situation  where  head  vibrations  do  not  cause  the  retinal  image  to  move  (Lee  &  King,  1971;  Velger,  1998, 
pp.  208,  232).  This  head  vibration  will  cause  visual  performance  to  decline.  For  instance,  acuity 
decrements  and  errors  in  aiming  increase  by  a  factor  of  10  during  head  vibration  (from  an  accuracy  of  0.1 
deg  without  head  vibration  to  1.0- 1.5  deg  with  head  vibration).  The  worst  performance  occurs  at  a 
frequency  range  of  3.2-5  Hz  (Velger,  1998,  pp.  216;  Wells  &  Griffin,  1987,  1988).  Below  1  Hz,  users 
can  make  smooth  pursuit  eye  movements  to  compensate  (Huddleston,  1970;  Velger,  1998,  pp.  216). 

One  remedy  for  an  inappropriate  VOR  response  while  wearing  a  HMD  is  to  measure  the 
involuntary  head  movements  with  a  head  tracker  and  shift  the  image  on  the  display  with  the  same 
magnitude,  but  in  a  direction  opposite  to  the  head  movement  (and  in  the  same  direction  as  the  VOR). 

This  method  attempts  to  stabilize  the  retinal  image  and  mimic  viewing  within  an  environment-centered 
frame  of  reference  (Velger,  1998,  pp.  225;  Young,  1976).  Because  most  head  motions  occur  within  the 
range  of  2-10  Hz,  the  head-position  measurement  computations  should  be  carried  out  at  a  rate  of  120-240 
Hz  (Velger,  1998,  pp.  227,  232;  see  Ljung,  Morf,  &  Falconer,  1978;  Merhav  &  Velger,  1991).  However, 
this  method  is  relatively  inaccurate  and  may  not  always  be  capable  of  distinguishing  voluntary  head 
movements  from  involuntary  head  movements  (Velger,  1998,  pp.  224;  Wells  &  Griffin,  1984,  1987). 

Accurate  measurement  of  head  position  is  challenging  because  head  movements  may  exceed 
angular  velocities  of  several  hundreds  deg/sec  and  accelerations  of  up  to  several  thousand  deg/sec^  (Wells 
&  Haas,  1990).  However,  these  values  are  at  the  extreme  end  of  head  movement  velocities  and 
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accelerations.  For  example,  Rash  et  al.  (1998)  found  that  97%  of  all  head  movements  of  AH-64  pilots 
were  between  0  and  120  deg/sec.  When  measuring  head  movements,  one  important  issue  is  the  temporal 
delay  and/or  lag  between  actual  head  position  and  update  of  the  displayed  imagery  (i.e.,  rendering 
latency).  So  &  Griffin  (1992)  found  that  lags  on  the  order  of  80-100  msec  between  head  movements  and 
corresponding  shift  of  displayed  imagery  produced  sensory  conflict  between  proprioception  and  visual 
information.  This  degree  of  temporal  delay  or  lag  can  produce  perceptual  confusion,  nausea,  and 
disorientation  when  viewing  a  simulated  out-the-window  view  (So  &  Griffin,  1992;  Velger,  1998,  pp. 

145,  171).  With  sufficiently  long  temporal  lags,  the  user's  perception  of  anticipated  flight  path  and 
vehicle  yaw  rate  become  impaired  (Grunwald  &  Kohn,  1994).  Moreover,  Jagacinski  and  Flach  (2003) 
note  that  phase  lags  set  up  a  stability  limit  on  the  forward  loop  gain  of  a  control  system,  with  larger  time 
delays  producing  decreasing  range  of  gains  that  will  yield  stable  control. 

With  long  time  delays,  there  may  be  no  stable  gain  that  produces  a  rapid  head-tracker  response 
and  a  proportional  control  system  may  not  work  (pp.  99).  According  to  Keller  and  Colucci  (1998),  the 
lag  between  the  image  displayed  and  actual  head  orientation/position/location  can  be  the  greatest  obstacle 
in  creating  effective  HMDs.  They  recommended  that  for  see-through  augmented  reality  HMDs  and 
targeting  applications,  the  lag  should  be  no  greater  than  16  msec.  Padmos  and  Milders  (1992)  suggest 
that  for  immersive  displays,  the  latency  of  update  should  be  less  than  40  msec.  To  achieve  these  values, 
one  may  need  to  increase  the  sampling  rate  or  computation  rate  of  head  position,  use  a  prediction 
algorithm  to  predict  head  position  at  some  future  point  in  time,  or  use  auxiliary  acceleration 
measurements  (e.g.,  accelerometers  or  gyroscopes  mounted  on  a  helmet;  Velger,  1998,  pp.  172;  Merhav 
&  Velger,  1991).  Olano,  Cohen,  Mine,  and  Bishop  (1995)  discussed  techniques  for  combating  rendering 
latency  by  making  the  rendering  latency  equal  one  NTSC  field  (1  video  frame,  or  16.7  msec  for  a  typical 
60  Hz  frame  rate)  and  by  rendering  the  pixels  in  a  scanned  display  based  on  position  in  the  scan. 

During  head  movements,  users  typically  move  their  eyes  as  well  as  their  heads  such  that  the  eye 
movement  leads  the  head  movement  and  there  is  an  angular  offset  between  eye  orientation  and  head 
orientation  (Velger,  1998,  pp.  173;  Robinson,  Koth,  &  Ringenbach,  1976).  Thus,  more  accurate 
measurements  of  line  of  sight  are  available  when  eye  movements  are  measured  as  well  as  head 
movements  (Velger,  1998,  pp.  173).  Some  current  eye  trackers  may  operate  at  the  speeds  and  accuracies 
required  for  an  eye-tracked  HMD.  In  a  survey  of  current  eye  tracking  technology  (Latham,  2004)  several 
eye  tracker  models  were  identified  with  accuracy  of  0.5  degrees  or  less  and  a  measurement  delay  less  than 
16  msec.  However,  many  of  these  eye  trackers  were  not  suitable  for  use  in  combination  with  an  HMD. 

Of  the  1 1  eye-tracking  manufacturers  surveyed,  only  one  was  identified  as  having  current  eye  tracking 
technology  suitable  for  use  in  combination  with  an  HMD.  Two  additional  companies  stated  that  HMD 
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compatibility  could  be  achieved  with  some  modification.  The  ability  to  integrate  eye  tracking  with  an 
HMD  depends  on  the  design  of  the  HMD.  For  some  HMD  designs,  it  may  not  be  possible  to  mount  a 
sensor  or  light  source  in  the  correct  position  necessary  for  measuring  eye  movements.  For  purposes  of 
simulating  an  environmental  frame  of  reference  with  a  head-centered  frame  of  reference,  the 
measurement  of  eye  movements  is  not  necessary  because  in  both  cases  the  pattern  of  retinal  stimulation 
would  change  with  an  eye  movement.  Eye  tracking  may  be  useful  for  reducing  image  generator 
requirements  through  the  use  of  an  eye  tracked  high-resolution  inset  within  a  lower  resolution  background 
image.  Reduction  of  IG  requirements  will  be  a  significant  issue  for  high-resolution  displays,  particularly 
when  portability  is  a  major  concern.  Latham  (2004)  determined  that  an  electronically  slewed  inset  using 
current  eye  tracking  technology  provided  a  feasible  solution  to  reducing  IG  requirements.  However, 
previous  implementations  of  eye-tracked  insets  (mechanical  rather  than  electronic)  were  generally  not 
perceived  to  have  been  satisfactory  (although  an  evaluation  by  Peters  &  Turner,  1992  indicated 
otherwise). 

CURRENT  HMD  TECHNOLOGY 

A  large  number  of  HMD  products  are  currently  on  the  market  that  range  in  price  from  hundreds 
of  dollars  to  tens  of  thousands  of  dollars.  Monocular  HMDs,  which  are  significantly  less  complex  in 
comparison  to  binocular  HMDs,  cost  substantially  less  than  binocular  HMDs.  Table  1  below  provides  a 
list  of  a  number  of  currently  available  HMDs,  their  manufacturers,  as  well  as  some  specifications  and 
pricing.  This  is  not  a  complete  list  and  a  number  of  other  manufacturers  such  as  Sony,  Canon,  Olympus, 
etc.  also  make  HMDs.  A  more  complete  list  of  HMDs,  which  includes  both  currently  available  devices 
and  discontinued  items,  is  maintained  by  StereoSD  at  http ://www. stereo3 d. com/hmd.htm#chart .  HMDs 
developed  specifically  for  use  in  aircraft,  such  as  the  JHMCS  (Vision  Systems  International)  are  also  not 
listed  here.  These  systems  typically  require  specialized  equipment  and  would  require  significant 
modification  for  use  in  simulation  and  training  applications.  The  products  that  are  listed  require  only 
standard  VGA  or  DVI  inputs  and  are  therefore  compatible  with  PC-based  image  generators  (PC-IGs) 
commonly  used  for  simulation. 
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Table  1.  Commercially  available  Head-mounted  displays  in  2005.  (All  photos  used  with  permission.) 


Commercially  available  Head-mounted  Displays  (2005) 

Photo 

Model 

Manufacturer 

Type 

FOV 

Display  Type  & 
Pixel  Format 

List  Price 

ProView  SR80 

Kaiser  Electro- 
Optics 

Binocular, 

Occluding 

63  degrees 
(H)/100%  overlap 

FLCoS,  SXGA 

$27,000 

Pro  View  SR100 

Kaiser  Electro- 
Optics 

Binocular,  Non¬ 
occluding  (25% 
transmissive) 

100  degrees  (H) 

FLCoS,  SXGA 

$67,000 

d 

ProView  S035 

Kaiser  Electro- 
Optics 

Monocular, 

Occluding 

28  degrees  (H) 

OLED,  SVGA 

$8500/$9500 

Advanced  HMD 

LSCommunications 

Binocular,  Non¬ 
occluding  (60% 
transmissive) 

100  degrees  (H)/ 
30  degree  overlap 

FLCoS,  SXGA 

NA 

1* 

LiteEye  400 

LiteEye 

Monocular, 

Occluding 

38  degrees 

OLED,  SVGA 

$1,599 

LiteEye  500 

LiteEye 

Monocular, 
Occluding  or  Non- 
Occluding 

28  degrees 

OLED,  SVGA 

$2,895 

SV-6 

MicroOptical 

Monocular,  Non¬ 
occluding 

16  degrees  (H) 

LCD,  VGA 

$1,995 

m 

Nomad  ND2500 

Microvision 

Monocular,  Non¬ 
occluding 

23  degrees  (H) 

Laser,  SVGA 

$6,000 

nVisor  SX 

NVIS 

Binocular, 

Occluding 

60  degree 
(diagonal)/100% 
overlap 

FLCoS,  SXGA 

$23,900 

nVisor  ST 

NVIS 

Binocular,  Non- 
Occluding  (50% 
transmissive) 

50  degree 
(diagonal)/100% 
overlap 

FLCoS,  SXGA 

$34,900 

ImT 

DataVisorSO 

nVision 

Binocular, 

Occluding 

120  degrees 
(H)/50%  overlap 

CRT,  VGA  or 
SXGA 

NA 

DataVisor 

nVision 

Binocular, 
Occluding  or  Non¬ 
occluding 

78  degrees  (H) 
w/50%  overlap:  52 
degrees  w/100% 
overlap 

CRT,  VGA  or 
SXGA 

NA 

SUMMARY  OF  HMD  CHARACTERISTICS 

Table  2  below  provides  a  summary  of  specifications  for  a  monocular  HMD  integrated  with  the 
AFRL  M2DART  for  OBS  training  applications.  Table  3  below  provides  a  summary  of  specifications  for 
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a  binocular  HMD  for  deployable  OTW  training  applications.  These  specifications  are,  wherever  possible, 
based  on  the  published  literature  related  to  perceptual  issues  and  human  factors  research  on  HMDs. 

HMD  characteristics  of  interest  are  listed  in  column  1  of  each  table.  Minimum  and  desirable 
specifications  are  given  in  columns  2  and  3,  respectively.  The  desired  specifications  do  not  necessarily 
reflect  current  engineering  limitations  for  each  of  the  characteristics  listed  but  provide  goals  for 
technology  development  for  HMDs.  Minimum  requirements  are  also  quite  stringent.  However,  many 
currently  available,  state-of-the-art  HMDs  approach  the  specifications  listed  in  this  column.  These 
represent  shorter-term  goals  for  HMDs  for  the  listed  requirements.  The  final  two  columns  of  each  table 
note  whether  each  listed  specification  has  been  clearly  established  and  whether  additional  technology 
development  is  needed  to  meet  it.  Additional  advances  in  technology  are  required,  for  example,  before  a 
field  of  view  matching  that  of  the  human  visual  system  and  a  resolution  of  1  arcmin/pixel  can  be 
achieved.  Additional  perceptual/human  factors  research  may  be  needed  to  better  define  specifications 
that  are  noted  as  not  clearly  established. 

Table  2.  Specifications  for  monocular  OBS  HMD  applications. 


Monocular  OBS  HMD  | 

Characteristic 

Desired  Specification 

Acceptable  Specification 

Clearly  Established  Criteria 

Technology  Development 
Required 

Accommodative  demand 
(focus  distance) 

Match  to  average  distance  (in 
diopters)  of  display  screen: 
0.905  diopters  for  M2DART^^^ 

Distance  in  diopters  +/-  0.1 
diopters 

Yes 

No 

Ocular  luminance  difference 

Isoluminance  in  left  and  right 
eyes 

10-25%  difference  between  left 
and  right  eye^^^ 

Yes 

No 

HMD  scene  content 

Unknown 

Minimize  occurrence  of 
binocular  rivalry* 

No 

Unknown 

Display  luminance 

70  fL* 

30  fL** 

Yes 

No 

Display  Black  Level 

0.001  fL^ 

0.1  fL** 

Yes 

Yes 

Resolution 

Application  dependent 

Match  to  deployed  equipment 
for  training  purposes 

N/A 

N/A 

Field  of  View 

Application  dependent 

Match  to  deployed  equipment 
for  training  purposes 

N/A 

N/A 

^  Winterbottom,  Patterson,  Pierce,  Covas,  &  Winner  (2005);  ^  Self  (1986) 

*lt  is  unknown  at  this  time  the  extent  to  which  rivalry  occurs  for  these  conditions  or  how  specifically  to  mitigate  against  rivalry,  however  Hershberger  &  Guerin 
(1975)  showed  that  ambient  scene  complexity,  HMD  resolution,  HMD  luminance,  HMD  accommodation  distance,  and  ambient  scene  luminance  were  significant 
factors;  *  Expected  M2DART  maximum  luminance  with  Laser  Display;  **  M2DART  maximum  luminance  with  CRT  displays;  ^  Approximate  CRT  black  level; 
^^Minimum  expected  OTW  scene  luminance 
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Table  3.  Specifications  for  binocular  OTW  HMD  applications. 


Binocular  HMD 

Characteristic 

Desired  Specification 

Minimum  Specification 

Clearly  Established  Criteria 

Technology  Development 
Required 

Display  Pixel  Hold  Time 

<  1  msec* 

<  8  msec** 

No 

Yes 

Display  Color  Breakup 

<  0.3  msec  pulse  duration*** 

<  2.7  msec  pulse  duration**** 

No 

Yes 

Refresh  Rate/Update  Rate 

120  Hz^^-^ 

60  Hz^^ 

No 

Yes 

Field  of  view  (flight 
simulation/motion  control) 

200  degrees^ 

127  degrees^^^ 

No 

Yes 

Resolution 

1  arcmin/pixel 

2.25  arcmin/pixel* 

Yes 

Yes 

Transmissivity 

60% 

50% 

No 

No 

Vertical  Misalignment 

2  aremin^^^ 

5  aremin^^^ 

Yes^ 

No 

Horizontal  Misalignment 

0  divergence,  3.4  areminutes 
convergence^^^ 

3.4  areminute  divergence,  8.6 
areminutes  convergence^^^ 

Yes^ 

No 

Rotational  Misalignment 

0 

3.4  areminutes^^^ 

Yes^ 

No 

Binocluar  overlap 

40  degrees^^^ 

>  25  degrees^"^^ 

No 

Yes 

Vergence/Accommodation 

imbalance 

Accommodation  matched  to 
vergence  angle^^^ 

+/-  0.1  diopters^®^ 

No 

No 

Eye  Relief 

50  mm 

25  mm^^^ 

Yes 

Yes 

Exit  Pupil 

15  mm 

10  mm^^^ 

Yes 

Yes 

System  Lag  (head 

tracker  +  IG) 

<  1 6  msec^^^ 

<  16  msec^^^ 

No 

Yes 

Weight/  center  gravity  (eg) 
(including  helmet  or  headband) 

<  3  lbs  with  eg  <  2  inches  in 
front  of  y-axis  of  head® 

<  4  lbs  with  eg  <  2  inches  in 
front  of  y-axis  of  head® 

No 

Yes 

Vruk  &  Runnings  (1989);  ^Self  (1986);  ^Grigsby  &  Tsou  (1994);  Vruk  &  Longridge  (1984);  ^Shibata  (2002);  ®Yeh  and  Silverstein  (1990);  ^Keller  and  Colucci 
(1998);  ®Melzer  &  Moffitt  (1997) 

*untested  but  based  in  part  on  data  of  Winterbottom  et  al  (2004)  may  reduce  motion  blur  by  roughly  90%;  **untested  but  based  on  data  of  Winterbottom  et  al 
(2004)  may  reduce  motion  blur  by  roughly  50%;  ***untested  but  based  on  equation  provided  by  Post  et  al  (1998)  should  prevent  noticable  color  break-up  for 
speeds  up  to  approximately  160  degrees/sec;  ****untested  but  based  on  equation  provided  by  Post  et  al  (1998)  should  prevent  noticable  color  break-up  for 
speeds  up  to  approximately  7  degrees/sec;  ^Partly  an  IG  requirement,  however  refresh  rate  should  match  update  rate  (Lindholm  &  Martin,  1993);  ^^Current 
display  refresh/update  rate,  display  refresh  rate  should  match  IG  update  rate  (Lindholm  &  Martin,  1993);^Matching  field  of  view  of  human  visual  system;^^Air 
Combat  Command  requirement;  *HDTV  resolution  assuming  current  M2DART  field  of  view;  However,  may  be  based  on  optical  systems  other  than  HMDs  (e.g. 
binoculars) 

In  our  view,  both  significant  technology  advances  and  additional  perceptual  research  are  needed, 
to  develop  an  HMD  for  deployable,  full  field  of  view,  OTW  training  applications.  Although  HMDs  are 
one  approach  to  developing  a  deployable  OTW  display  for  advanced  aircraft  simulation  and  training 
systems,  user  acceptance  will  be  a  critical  hurdle  and  may  depend  on  meeting  all  of  the  specifications 
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listed  in  the  table,  rather  than  just  a  few.  Developing  solutions  to  the  limitations  in  current  HMDs  will 
require  a  solid  understanding  of  many  of  the  perceptual  issues  described  in  this  report,  as  well  as 
continued  technology  development  in  many  of  the  areas  described  here  and  summarized  in  Tables  2  and 

3. 

FUTURE  RESEARCH 

In  addition  to  several  technology  issues  that  are  in  need  of  further  development  (reduction  of 
motion  artifacts  in  digital  displays,  decreasing  color  break-up  in  field  sequential  displays,  increasing 
display  resolution,  increasing  field  of  view  for  near  to  eye  applications,  and  decreasing  head-tracking  lag), 
there  are  also  several  perceptual  issues  that  need  to  be  resolved  when  attempting  to  simulate  off-bore 
sighting  or  OTW  viewing.  These  issues  are  the  following: 

(1)  Binocular  Rivalry.  In  simulating  off-bore  sighting,  one  eye  views  the  HMD  imagery  and  the 
simulator  imagery  combined  while  the  other  eye  views  simulator  scenery  alone.  In  this  case,  the 
existence  of  interocular  differences  in  stimulus  characteristics  may  produce  binocular  rivalry.  It  is 
important  to  determine  whether  rivalry  occurs  under  these  viewing  conditions  and,  if  so,  determine  what 
stimulus  characteristics  may  reduce  its  occurrence.  In  simulating  OTW  viewing,  the  two  eyes  will  view  a 
binocular  HMD  that  depicts  an  out-the-window  scene  of  an  ongoing  mission.  In  this  case,  the  existence 
of  small,  unintended  interocular  differences  in  stimulus  characteristics  may  occur  due  to  imprecision  of 
mounting  and  alignment,  which  may  produce  rivalry  or  other  unwanted  perceptual  effects.  For  example, 
it  is  known  that  small  interocular  differences  in  stimulus  size  can  produce  the  perception  of  stereoscopic 
slant  (Ogle,  1952).  Additionally,  it  may  be  desirable  to  simulate  both  OTW  viewing  conditions  and 
JHMCS  type  symbology  using  a  binocular  HMD.  In  this  case,  interocular  differences  conducive  to 
rivalry  may  also  be  produced.  It  is  therefore  important  to  determine  which  interocular  differences 
produce  rivalry  or  other  unwanted  effects  and  whether  training  or  practice  can  help  mitigate  such  effects 
(for  example  see  Rash,  Mozo,  McEntire  and  Licina,  1996  for  a  discussion  of  limits  of  interocular 
tolerance). 

(2)  Accommodation- Vergence  Mismatch.  When  simulating  off-bore  sighting  with  a  monocular 
HMD  worn  in  a  flight  simulator,  or  simulating  OTW  viewing  with  a  binocular  HMD,  the  stimulus  to 
accommodation  will  be  the  imagery  of  the  HMD  and/or  display  screen  of  the  simulator.  However,  when 
the  user  changes  his  or  her  vergence  angle  to  binocularly  view  a  virtual  target  in  the  synthetic  scene,  the 
vergence  angle  can  be  mismatched  relative  to  accommodative  demand.  There  are  three  potential  effects 
of  accommodation- vergence  mismatch:  (1)  eye  strain  or  visual  discomfort;  (2)  blurring  of  imagery 
(depending  upon  display  brightness  and  contrast);  and  (3)  misperceived  size,  depth,  speed,  and/or 
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distance  via  the  operation  of  perceptual  constancy.  The  effects  of  accommodation/vergence  mismatch  on 
visual  discomfort,  blurring  of  imagery,  and  perceived  size,  depth,  speed  and  distance  should  be  examined. 

(3)  Field  of  view.  For  simulating  both  off-bore  sighting  and  OTW  viewing,  field  of  view  can 
affect  perception  (e.g.,  perceived  size,  distance)  as  well  as  performance.  This  is  because,  in  part,  field  of 
view  determines  the  relative  amount  of  stimulation  of  the  parvocellular  visual  pathway  (shape,  fine  spatial 
acuity,  color  perception)  versus  the  magnocellular  pathway  (motion  processing,  location  information).  It 
is  important  to  determine  what  size  of  field  of  view  is  needed  and  how  that  size  will  affect  other  important 
factors  such  as  binocular  overlap  and  resolution. 

(4)  Binocular  overlap  and  convergent  versus  divergent  displays.  In  simulating  OTW  viewing,  it 
is  important  to  determine  the  perceptual  implications  of  using  either  convergent  or  divergent  binocular 
displays.  In  particular,  one  should  determine  how  the  convergent  versus  divergent  systems  relate  to  half¬ 
occlusions,  visual  suppression,  luning,  and  field  of  view,  and  whether  one  type  of  display  is  better  than 
the  other  in  terms  of  less  luning  or  whether  a  full-overlap  display  should  be  considered.  See  Rash,  Mozo, 
McEntire,  and  Licina  (1996)  for  discussion  of  luning.  However,  it  may  be  more  important  to  match 
accommodative  demand  with  vergence  to  reduce  eyestrain  (Shibata,  2002).  For  extended  viewing 
conditions  such  as  those  encountered  during  simulation  and  training  exercises,  it  will  be  important  to 
reduce  eyestrain  and  fatigue  as  much  as  possible. 

(5)  Head  tracking  of  voluntary  head  movements.  In  simulating  an  out-the-window  view,  the 
display  imagery  must  be  shifted  in  a  direction  opposite  to  voluntary  head  and  pursuit  eye  movements  to 
mimic  an  environment-centered  frame  of  reference.  One  problem  is  that  the  time  required  to  measure  the 
head  movements  with  the  head  tracking  system  and  update  the  display  creates  a  temporal  lag  that  impairs 
perception  and  cognition  and  also  places  constraints  on  the  gain  of  the  head  tracking  system  (which 
potentially  decreases  the  usefulness  of  the  head  tracker  for  measuring  large  voluntary  head  movements). 

It  is  therefore  important  to  investigate  the  relationships  among  head-tracking  accuracy,  lag,  and  size  of 
head  movement  so  as  to  determine  an  acceptable  set  of  parameters. 

(6)  Head  tracking  of  involuntary  head  movements.  If  mission  training  or  rehearsal  utilizing  an 
HMD  is  conducted  in  an  environment  that  causes  acceleration  or  vibration,  the  head  will  execute 
involuntary  head  movements  that,  in  turn,  elicit  a  compensatory  and  opposite  VOR  eye  movement.  The 
VOR  eye  movement  impairs  viewing  of  the  HMD  because  the  image  moves  with  the  head,  which  renders 
the  VOR  response  as  inappropriate.  One  approach  for  correcting  inappropriate  VOR  responses  while 
wearing  an  HMD  is  to  measure  the  involuntary  head  movements  with  a  head  tracker  and  shift  the  image 
on  the  HMD  by  the  same  magnitude  and  a  direction  opposite  to  the  head  movement  (and  in  the  same 
direction  as  the  eye  movement).  However,  a  difficulty  with  this  technique  is  that  the  head  tracker  may 
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not  be  accurate  enough  to  fully  compensate  for  the  VOR  response.  It  is  therefore  important  to  determine 
to  what  degree  HMDs  will  be  used  for  mission  training  and  rehearsal  in  environments  involving 
acceleration  or  vibration.  If  those  conditions  are  anticipated,  then  the  relationships  among  head-tracking 
accuracy,  lag,  and  size  of  head  movement  should  be  investigated  so  as  to  determine  whether  small 
(involuntary)  head  movements  can  be  measured  with  an  acceptable  degree  of  accuracy  and  lag. 
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APPENDIX 


ACC 

Air  Combat  Command 

AFRL 

Air  Force  Research  Laboratory 

AMLCD 

Active  matrix  liquid  crystal  displays 

ATD 

Advanced  Technology  Demonstration 

CODS 

Conjugate  Optical  Display  System 

CRT 

Cathode  Ray  Tube 

DTP 

Digital  Light  Processing 

DMD 

Digital  Micro-Mirror  Display 

DMO 

Distributed  Mission  Operations 

DOF 

Degrees  of  Freedom 

FLCoS 

Ferro-Electric  Liquid  Crystal  on  Silicon 

FLIR 

Forward  Looking  Infrared 

FOV 

Field  of  View 

GLV 

Grating  Light  Valve 

HMD 

Head-Mounted  Display 

HMPD 

Head-Mounted  Projection  Display 

HUD 

Head-Up  Display 

IHADSS 

Integrated  Helmet  and  Display  Sighting  System 

JHMCS 

Joint  Helmet  Mounted  Cueing  System 

LCoS 

Liquid  Crystal  on  Silicon 

LED 

Light  Emitting  Diode 

M2DART 

Mobile  Modular  Display  For  Advanced  Research  And  Training 

OBS 

Off-Bore  Sight 

MEMS 

Micro-Electro-Mechanical  System 

OLED 

Organic-Light-Emitting-Diode 

OTW 

Out  the  Window 

PDA 

Personal  Digital  Assistant 

VOR 

Vestibulo-Ocular  Reflex 

VRD 

Virtual  Retinal  Displays 
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